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ABSTRACT 

Interim estimates were made of program impacts on 
earnings and employment of adults and out-of-school youths during the 
first 18 months after their acceptance into employment and training 
programs funded under Title IIA of the Job Training Partnership Act 
(JTPA). In the national study, 20,601 JTPA applicants in 16 service 
delivery areas were randomly assigned to treatment (allowed access to 
the program) or control groups. Analysis was based on a subsample of 
17,026 participants interviewed 18 months after assignment. Different 
service strategies (classroom training, on-the-job training/job 
search assistance, and other services) were used for four subgroups: 
adult men, adult women, cut-of-school male youths, and out-of-school 
female youths. Findings included the following: (1) Title IIA had 
generally positive effects on employment and earnings of adults in 
the treatment group; (2) it had little or no effect on average 
earnings of female youths, actually reduced earnings of male youths, 
and had no significant effect on youth employment rates; (3) almost 
all the negative impact on male youths* earnings was among those who 
had been arrested between age 16 and random assignment (257. of the 
subgroup); (4) earnings impacts reflected amount of time worked and 
how much was earned per hour; and (5) sites did not differ 
significantly in degree of effect on earnings. It was concluded that 
JTPA Title IIA is helping raise the earnings of many participants, 
especially adults. It is possible that adult/youth differences are 
the result of age alone. Ways to improve program effects for those 
groups negligibly or adversely affected by the program will require 
experimentation with a range of alternative service strati^jies and 
rigorous evaluation of their impacts. (Appendices contain the 
following: comparison of study sample with nationwide Title IIA 
participants; comparison of baseline characteristics of treatment and 
control groups; data sources; sample definition and impact evaluation 
method; comparison of data from first follow-up survey and state 
unemployment insurance agencies; analysis of nonenrol lees ; estimation 
of impacts on latent wage rates; and exhibits on out-of-school youth. 
There are 67 references and 156 data tables.) (YLB) 



i> 

CO 

^ A draft report to the 

(m) V ,S. Department of Labor 

Jjg^ Larry L. Orr, Project Director 



Associaie> Iik'. 



The National JTPA Study 



Title II-A Impacts on 
Earnings and Employment 
at 18 Months 



Howard S, Bloom 
Larry L. Orr 
George Cave 
Stephen H. Bell 
Fred Doolittle 



o 

ERIC 



U 8 DCfAPTTMCNTOr EOUCATtON 
0«>ce Oi EduC*lK>n«l R«s««rch snO »mpfOv«m«>ni 
EDUCATIONAL RESOURCES INFORMATION 
' CENTER (ERIC) 



{/inn (Jocument h« Wjen fepfO<3i>c«3 •« 
ttom fne parson or oro.n.l.t.on 
onginsUng it 
□ Minor .h«ng«s have Wje'^ m.cJe to .mprove 
repfo<JoctioA quality 



Points o< v.«* or opinions Sl«te<J 'n th.S dOCu 
m«nt do not n.cessar.ly represent o«.ci»i 
OERl posilton or ooUcy 



Petliesda, Maryland 



Abt Associates Inc. 



Wit)i the assisiancf of' 
^fVinston Lin 
Johannes Bos 
Cyril Toussaini 
Robert Kornfeld 



January 1993 



. r\ I X 



2 eESTf0"7£MlE 



The National JTPA Study 



Lany L. Orr, Project Director for impact and cost-benefit analyses, and 

Co-Principal Investigator 
Fred Doolittle, Projea Director for implementation of the study and process analysis 
Howard S. Bloom, Co-Principal Investigator 
Judith M. Gueron, Co-Principal Investigator 

The National JTPA Study: Tule II-A Impacts on Earnings and Employment at 28 Months 
was produced under Contract Number 99-64)803-77-O68-01 with the U.S. Depailment of 
Labor (DOL). The statemenU contained in this report are solely those of the authors and do 
not necessarily reflect the views of staflFmcmbers at DOL or the 16 study sites. Abt Associates 
Inc. assumes full responsibility for the accuracy and completeness of the information 
prticnted here. 



Published in the United States of America by 
Abt Associates Inc. 

Hampden Square, Sizite 600 
4800 Montgomery Lane 
Beihesda, Maryland 20814-5314 
(301)913-0500 



Corporate offices: 
Abt Associates Inc. 
55 Wheeler Street 

Cambridge, Massachusetts 02138-1 168 
(617) 492-7100 



Library of Congress Cataloging-in-Publication Data 

The national JTPA stud>'.; Title II-A impacts on earnings and 
employment at 18 months / Howard S. Bloom. . .(et al.] ; with the 
assistance of Winston Lin. . .(et al.]. 
p. cm. 

Includes bibliographical references. 
ISBN (invalid) 0-8901 1-622-4 (pbk.) 

1. Occupational training— United States— Evaluation. I. Bloom, 
Howard S. 

HD5715.2.N44 1992 

33 l,26'92'0973^c20 92-14239 

CIP 









Contnct Manager 




Management RevieweFw/ 



Acknowledgments 



T'HE National JTPA Study was conceived and recommended to the U.S. Department of 
Labor (DOL) by the Job Training Longitudinal Survey Advisory Panel; chaired b\- Ernst 
Stromsdorfer, the panel also included Howard S. Bloom, Robert Boruch, Michael E. Borus, 
Judith M. Gueroa Alan Gustman, Peter Rossi, Fritz Scheuren, Marshall Smith, and Frank 
Stafford. The pathbreaking decision by DOL's Emplo\Tnent and Training Administration to 
implement a classical experimental evaluation of an ongoing national program was also 
influenced b\' the strong recommendation of the National Research Council Committee on 
Youth EmploN-ment Programs, chaired by Robinson G. Hollister, Jr., that random assignment 
designs be used to evaluate emplo\Tnent and training programs. 

We are particularly grateful to Ra\Tnond J. Uhalde, Administrator, Office of Strategic 
Planning and Policy Development, EmploNment and Training Administration, who created 
the project and ensured that it was sustained throughout the lengthy period required to bring 
it to fruition. Other staff members for the Department of Labor who played a major role in 
formulating and implementing the project w^ere David Lah, John Heinberg, Karen Greene, 
Dan Ryan, and Gerald Gunderson. 

Thanks are also due for the insightful advice offered at key junctures by the Advisor\' Panel 
for the National JTPA Study: Isabel V. Sawhill, chair; John Bishop, Nancy Bross, Gar\- 
Burtless. Richard Hayes, Christoper King, MariKn E. Manser, Ronald L. Oaxaca, Gary- 
Reed, and Ernst Stromsdorfer. 

This '■eport represents the ideas and efforts of man\' persons from different organizations 
who worked together as an unusualh' effective team. The co-principal investigators for the 



ERIC 



VI • JTPA 18-MONTH IMPACTS / ACKNOWLE.^GMENTS 



National JTPA Study impact and cost-benefit analyses are project director Lam* L. Orr. Abt 
Associates Inc.: Howard S. Bloom, New York Universit>'; and Judith M. Gueron. Manpower 
Demonstration Research Corporation (MDRC). Fred Doolittle, MDRC, is project director 
for implementation of the study and process analysis. Stephen H. BelL Abt Associates Inc.. 
and George Cave, MDRC, are the senior economists responsible for the impact analyses for 
adult women and youths, respectiveh*. 

Other members of the research team who made extensive contributions to the 1 8-month 
impact analysis include Winston Lin and Robert Komfeld of Abt Associates Inc.; Johannes 
(Hans) Bos and CvTil Toussaint of MDRC; and Wendy L. Campbell of Research Findings 
in Print. Hans and Winston provided substantial input to the analysis that produced the 
findings in the report and wTOte portions of its appendices. In addition, Winston made major 
methodological contributions to the analysis and Hans, with WendN\ designed the e.diibits and 
this document. Hans, Winston, and Cyril did the computations for the report. Robert 
contributed much of the analysis for one of the appendices. Wendy served as the editor of the 
report and wove together the man\' threads from many authors to make it speak with 
one voice. Bob Komfeld helped wTite Appendix E, and Cyril Toussaint helped provide the 
findings for Chapter 6. 

Major roles in crafting the report were played by both Judith M. Gueron, president of 
MDRC and co-principal investigator of the study and Stephen Kennedy, Chief Social Scientist 
of Abt Associates Inc. and technical reviewer for the stud\'. Judy and Steve eacii had a strong 
influence on the structure, content, and focus of the report. 

Other members of the research team who contributed to the report include Margorie 
Morrissey of the National Opinion Research Center (NORC), who served as survey director 
for the First Follow^-up Survey; Michelle Wood of Abt Associates, who supervised the 
collection and processing of the Background Information Forms; Robert Teitel, who super- 
vised the preparation of the 18-month data base, with the assistance of Kevin Ward, Peter 
Hurlej . Aniruddha Bonneijee. David Bell. Rhonda Byrnes, and Neelima Grover. all of Abt 
Associates: and Robin Weiss-Castro. Kristina Groner. and Kirsten Alexander who prepared 
the report under the supervision of Julie Masker. At MDRC James Kemple and John Wallace 
served as liaisons betw een the impact analysis and the implementation analysis; Colleen Galvin 
assisted in preparing the impact analysis for youths: and Robert Winthrop contributed to 
Chapter 3 of the report. 

The stud\' owes its existence to the commitment and energies of the staff at the 16 JTPA 
service deliverv- areas who not onK* agreed to participate, but also carried out the many tasks 
necessary- for implementation. Those who held the designated positions during the course of 
stud\* were: 



JTPA 18-MONTH IMPACTS / ACKNOWLEDGMENTS • VH 



Site 

Butte. Montana 

Cedar Rapids, Iowa 
Coosa Valley, Georgia 

Corpus Christi, Texas 

Decatur. Illinois 
Fort Wa\Tie, Indiana 
Heartland. Florida 

Jackson. Mississippi 

Jersey City, Ne\\ Jersey 

Larimer County, Colorado 
Marion. Ohio 

Northwest Minnesota 
Oakland, California 

Omaha, Nebraska 
Providence, Rhode Island 

Springfield, Missouri 



SDA^PIC Director 
Sue Mohr 

Robert Ballant>'ne 

James H. La\ton 
C. D. Rampley 

Irma Caballero 
Debra Seeger 

John Roark 

Steve Corona 

Jack Lyons 
Clifton Thomas, Jr. 

Beneta Burt 
Archester Hampton 

William Colon 
Anthony Corsi 
Jeny- DelPiano 

Neil Gluckmari 

Samantha Carroll 
Jill Navarrette 
Patrick Powell 

Gail Butenhoff 

Susan Caldwell 
Gay Plair Cobb 

Ola M. Anderson 
Fernando Lecouna III 

William D. Fomicola 
Robert Palumbo 
Ronald Perillo 

Chet Dixon 



Study Coordinator 

John Ilgenfritz 
Candi Watts 

Bonnie Pisarik 

Angeline Bedwell 
Gwen Del linger 

Billie O'Dowdv 



Jacque Matson 
Bett\- Lou Nault 
Alice Cobb 

Archester Hampton 

Keith Davis 
Judith Martin 

Joni Friedman 
Steven Pvles 



Ken Barborak 

Paulette Cathey 
Edna Davis 
Ralph Zackheim 

Karen Benson 

Ed Canner 
Bob Lonardo 

Mar\- Schacffer 



Contents 



Acknowledgments y 

List of Exhibit^ xi\ 

Executive Summary xxxiii 

The National JTPA Study xxxiii 

The 18-Month Impact Analysis xxxiv 

Overall Impacts on Earnings and Emplo\TOent, by Target Group xxxv 

Findings for Subgroups of Adult Women and Men xxxix 

Service Strategies Recommended xxxix 

Enrollment Rates and Duration, by Service Strategy Subgroup xl 

Services Received, by Service Strategj' Subgroup xli 

Impacts on Educational Attainment, by Service Strategy Subgroup xliii 

Impacts on Earnings, by Service Strategy Subgroup xliv 

ix 



ERIC 



i 



y. - JTPA iS-MOhTTH IMPACTS / TABLE OF COKTEKTS 



Impacts on Earnings, by Ethnicity and Barriers to EmploN-ment xlvn 

Sununary and Comparison with Previous Findings ' >^li>i 

Findings for Subgroups of Female and Male Out-of-School Youths li 
Service Strategies Recommended 

Enrollment Rates and Duration, by Service Strateg\' Subgroup lii 

Services Received, by Service Strategy Subgroup hii 

In pacts on Educational Attainment, by Service Strategy Subgroup li\ 

Impacts on Earnings, by Service Strategy' Subgroup Iv 

Impacts on Earnings, b\' Ethnicity, Barriers to Employment, and 
Reported Arrests 

Summary and Comparison wth Previous Findings l>^i 
Implications of the Findings •^i' 

1. Background: JTPA Title II-A Nationally, 

Previous Research, and the National JTPA Study 1 

The JTPA Title II-A Program Nationally 2 
Administration 

Services ^ 
Participants 

Penomiance ^ 

Recent Policy Amendments 7 

Previous Studies of Employment and Training Programs 8 

The Central Methodological Problem 8 

EarK- Studies ^ 



8 



rrPA 18-MONTH IMPACTS / TABLE OF CONTENTS • XI 



Second Generation Stuciies ^ 0 

Tie National CETA Evaluations 1 0 

Randomized Experiments 1 1 
Methodological Studies of Experimental and Comparison Group Techniques 14 

Recommendations of the National Academy of Sciences Committee on 

Youth Employment Programs 1 5 

Recommendations of the JTLS Research Advisory Panel 1 5 

The National JTPA Study in Brief 1 7 

Summar\' 1^ 

2. Preview: The 18-Month Impact Analysis 21 

Implementation of the Study Design 22 

Site Selection 22 

Client Intake and Random Assignment 25 

The Four Target Groups 2 7 

The Three Service Strategies 28 

Impact Estimates in the 18-Month Analysis 30 

The 1 8-Month Study Sample 30 

Impacts per JTPA Assignee 32 

Impacts per JTPA Enrollee 33 

The Incremental Impact of JTPA 35 

Impacts on Attainment of a High School Credential 35 

Impacts on Earnings, EmploNinenl, and Earnings Components 36 



'J 



ERIC 



• TTPA 18-MONTH IMPACTS / TABLE OF CONTEbTTS 

Data for the Report 

Context: Study Sites, 18-Month Study Sample, and 
Patterns of JTPA Enrollment and Service Receipt 

The 16 Study Sites 

Population Characteristics 

Economic Conditions 

Participant Characteristics 

Program Characteristics 

Agency Standards and Program Performance 

Services Received at the Study Sites 
A Profile of the 18-Month Study Sample 

Assignees, EnroUees, and Control Group Members 

Target Groups 

Subgroups Facing Selected Barriers to Emplo>Tnent 
Patterns of JTPA Enrollment and Service Receipt 
Service Strategies Recommended 

Differences in Employabilit>' across Service Strategy Subgroups 

Differences in JTPA Service Receipt across Target Groups 
and Service Strategy Subgroups 

Enrollment Patterns Over Time Among EnroUees and 
the Treatment Group Overal. 

JTPA Services Received by Treatment Group Nonenrollees 
Summar\' 

10 



JTPA IS-MOhTTH IMPACTS / TABLE OF CONTENTS • XIII 



4. Adult Women: JTPA Impacts at 18 Months 75 

Impacts on Earnings: Adult Women Overall 77 

Monthly Earnings Trends: Treatment Group and Control Group 77 

Enrolhnent Patterns over Time: Treatment Group 79 

Impacts on Earnings: JTPA Assignees and Enrollees 80 

Impacts on the Distribution of Earnings: JTPA Assignees 83 

Impacts on Emplojment: Adult Women Overall 85 

Impacts on the Components of Earnings: Adult Women Overall 87 

Impacts on Earnings, Employment, and Earnings Components: 

Adult Women Recommended for Each Service Strategj' 89 

Differences in Emplojinent and Training Services Received: 

Treatment and Control Groups wthin Each Service Strateg>' Subgroup 90 

Differences in Baseline Characteristics across Service 

StrategN' Subgroups: JTPA Assignees 100 

Monthly Earnings Trends: Treatment and Control Groups 

within Each Service Strategj' Subgroup 103 

Impacts on Earnings: JTPA Assignees and Enrollees in Each 

Service Strateg\' Subgroup 1 05 

Impacts on Employment: JTPA Assignees in Each Service 

Strategj' S ubgroup 1 08 

Impacts on the Components of Earnings; JTPA Assignees 

in Each Service Strategy Subgroup 1 10 

Impacts on Earnings: Adult Women in Selected Key Subgroups 113 

Ethnic Groups 1 15 

Subgroups Facing Selected Barriers to Employment 1 18 

Subgroups with Different Work Histories 120 



ERIC 



n 



XI\' • rrPA 18-MONTH IMPACTS / TABLE OF CONTENTS 



Subgroups with Different Public Assistance Histories 120 

Subgroups from Different Types of Households 1 2 1 

Age Groups 121 

Women Recommended for Job Search Assistance Only 122 

A Summar\' of Impacts, in the Context of Previous Research 122 

Adult Women Overall 1 22 

Women Recommended for Each Service Strategy^ 123 

Women in Selected Key Subgroups 130 

5. Adult Men: JTPA Impacts at 18 Months 133 

Impacts on Earnings: Adult Men Overall 133 

Monthly Earnings Trends: Treatment Group and Control Group 133 

Enrollment Patterns over Time: Treatment Group 135 

Impacts on Earnings: JTPA Assignees and Enrol lees 137 

Impacts on the Distribution of Earnings: JTPA Assignees 139 

Impacts on EmplovTnent: Adult Men Overall 141 

Impacts on the Components of Earnings: Adult Men Overall 143 

Impacts on Earnings, Employment, and Earnings Components: 

Adult Men Recommended for Each Service Strateg>' 145 

Differences in Emploxinent and Training Services Received: Treatment 

and Control Groups within Each Service Strategy Subgroup 146 

Differences in Baseline Characteristics across Service 

Strategy Subgroups: JTPA Assigne^^s 155 

Monthly Earnings Trends: Treatment and Control Groups 

within Each Service Strateg\' Subgroup 159 



12 

ERIC 



JTPA 18-MONTH IMPACTS / TABLE OF CONTENTS • XV 



Impacts on Earnings: JTPA Assignees and Enrollees in Each 
Service Strateg\' Subgroup 

Impacts on Emplo>Tnent: JTPA Assignees in Each Serv^ice 

Strateg>' Subgroup ' ^ ^ 

Impacts on the Components of Earnings: JTPA Assignees 

in Each Service Strateg>' Subgroup 164 

Impacts on Earnings: Adult Men in Selected Key Subgroups 166 

Ethnic Groups ^'^^ 

Subgroups Facing Selected Barriers to Employment 1 7 1 

Subgroups Employed or Not Employed upon Application to JTPA 172 

Subgroups Living in PubHc Housing or Not 1 72 

Men Recommended for Job Search Assistance Only 1 73 

A Summary of Impacts, in the Context of Previous Research 173 

Adult Men Overall ^'^^ 

Men Recommended for Each Service Strategy 1 79 
Men in Selected Key Subgroups 179 

Out-of-School Youths: JTPA Impacts at 18 Months 181 
Impacts on Earnings: Out-of-School Youths Overall and by Gender 1 83 

Monthly Earnings Trends: Treatment Group and Control Group 184 
Enrollment Patterns over Time: Treatment Group 1 86 

Impacts on Earnings: JTPA Assignees and Enrollees 1 86 

Impacts on Earnings: Youths and Young Adults 1 89 

Impacts on Earnings and Its Components: Female Youths 

Recommended for Each Ser\-ice Strategy 189 



13 



ERIC 



XVI • rrPA 18-MONTH IMPACTS / TABLE OF CONTENTS 



Differences in EmploNnient and Training Services Received: Treatment 

and Control Groups within Each Service Strategy Subgroup 191 

Differences in Baseline Characteristics across Service 

Strategy Subgroups: JTPA Assignees 197 
Impacts on Earnings: JTPA Assignees and Enrollees in 

Each Service Strategy' Subgroup 197 



Impacts on the Components of Earnings: JTPA 
Assignees in Each Service Strategy' Subgroup 



200 



Impacts on Earnings and Its Components: Male Youths 

Recommended for Each Service Strateg\' 204 

Differences in EmploNinent and Training Services Received: Treatment 

and Control Groups within Each Service Strategy' Subgroup 204 

Differences in Baseline Characteristics across Service 

Strateg\' Subgroups: JTPA Assignees 208 
Impacts on Earnings: JTPA Assignees and Enrollees in 

Each Service Strategy* Subgroup 212 
Impacts on the Components of Earnings: 

JTPA Assignees in Each Service Strategy Subgroup 214 
Impacts on Earnings: Female and Male Youths in 

Selected Key Subgroups 21^ 

Female Youths in Key Subgroups 220 

Male Youths in Key Subgroups 22 1 

A Summary of Impacts, in the Context of Previous Research 226 

Female Youths 226 

Male Youths 228 
Previous Experimental Findings on 

Programs for Out-of-School Youths 230 



14 



rrPA IS-MONTH IMPACTS / TABLE OF CONTENTS • XVII 



7. A Comparison of JTPA Impacts across Target Groups 
and Study Sites 



235 



Impacts on Earnings and Emplo\TOent Overall -36 

Impacts on Earnings: JTPA Assignees Overall -37 

Impacts on Employment: JTPA Assignees Overall 239 

Services Recommended and Received 240 

Service Strategies Recommended: Treatment Group 240 
JTPA Services Received: Treatment Group, 

by Service Strategy' Subgroup 24 1 
Services Received from Any Provider: Treatment and 

Control Groups, by Service Strategy Subgroup 244 

High School Attainment: Treatment and Control Groups 247 
Impacts on Earnings across Service Strategy Subgroups 249 
Impacts on 18-Month Earnings 249 
Impacts on Quarterly Earnings 2^ 1 

General Findings ^^2 
Impacts on the Components of Earnings 253 
Impacts on Earnings across Key Subgroups 236 
Ethnic Groups 258 
Subgroups Facing Selected Barriers to EmploNinent 259 
Impacts on Earnings across Study Sites 259 

The Influence of Selected Site and Baseline Characteristics 

on Program Impacts 26:> 
Possible Determinants of Impact 266 



1 



ERIC 



XVin • JTPA i 8-MONTH IMPACTS / TABLE OF CONTENTS 



Limitations of the Analysis ^68 
Findings 

A Summan- of Cross-Target Group and Cross-Site Comparisons 27? 

Target Groups 275 

Sites 274 

Summar>' 274 

Appendix A. A Comparison of JTPA Enrollees in the i8-Month Study Sample 

and theTitle II-A Participant Population Nationwide 277 

Appendix P, A Statistical Comparison of Treatment-Control 

Group Differences in Baseline Characteristics 287 

Appendix C. Data Sources for the 18-Month Impact Analysis 305 

Appendix D. Sample Deflnition and Impact Estimation Method 319 

Appendix E. A Comparison of Earnings, Employment, and Impact Estimates 
Based on Data from the First Follow-Up Survey and from 

State Unemployment Insurance Agencies 345 

Appendix F. Nonenrollees and an \nalysis of Their Participation in JTPA 367 

Appendix G. Estimation of Impacts on Latent Wage Rates 373 

Appendix H. Supplementary Exhibits on Out-of-School Youth 377 

References 397 



ERIC 



Exhibits 



Executive Summary 



Exhibit S. 1 Impacts on Total 18-Month Earnings and Employment: 
JTPA Assignees and Enrollees, by Target Group 

Exhibit S.2 Percentage Impacts on Total 18-Month Earnings 

and Its Components: JTPA Assignees and Enrollees, 
by Target Group 

Exhibit S.3 Service Strategies Reconunended: Adult JTPA 
Assignees, by Gender 

Exhibit S.4 Key JTPA Services Received by Treatment Group 

Members Who Were Enrolled in the Program: Adults, 
by Gender and Service Strategy Subgroup 

Exhibit S.5 Impacts on Attainment of a Training-Related High 
School Diploma or GED Certificate: Adult JTPA 
Assignees Who Were High School Dropouts, by Gender 

Exhibit S.6 Impacts on Quarterly and 18-Month Earnings: Adult JTPA 
Assignees, by Gender and Service Strategy Subgroup 

Exhibit S.7 Impacts on the 18-Month Earnings of Major Ethnic Groups: 
Adult JTPA Assignees, by Gender 



XXXVI 



xxxvin 



xl 



xlii 



xliii 



xlv 



xlviii 



17 



XX • JTPA 18-MONTH IMPACTS / LIST OF EXHIBITS 



Exhibit S.8 Impacts on the 18-Month Earnings of Subgroups 
Facing Selected Barriers to Emjoyment: Adults 
JTPA Assignees, by Gender 

Exhibit S.9 Service Strategies Reconunended: Out-of-School 
Youth JTPA Assignees, by Gender 

Exhibit S. 10 Key JTPA Services Received by Treatment Group 
Members Who Were EnroUcu in the Program: 
Out-of-School Youths, by Gender and Service 
Strategy Subgroup 

Exhibit S. 11 Impacts on Attainment of a Training-Related High School 
Diploma or GED Certificate: Out-of-School Youth JTPA 
Assignees Who Were High School Dropouts, by Gender 

Exhibit S. 12 Impacts on Quarterly and 18-Month Earnings: 

Out-of-School Youth JTTA Assignees, by Gender and 
Service Strategy Subgroup 

Exhibit S.13 Impacts on the 18-Month Earnings of Major Ethnic Groups: 
Out-of-School Youth JTPA Assignees, by Gender 

Exhibit S. 14 Impacts on the i8-Month Earnings of Subgroups Facing 
Selected Barriers to Employment: Out-of-School Youth 
JTPA Assignees, by Gender 

Exhibit S. 15 Impacts on the 18-Month Earnings of Subgroups With 
and Without a Previous Arrest: Out-of-School Youth 
JTPA Assignees, by Gender 



xlix 



liii 



Iv 



Ivi 



lix 



Ix 



Ixi 



Chapter 1 

Exhibit 1.1 



Summary of Estimated CETA Impacts on Annual Earnings, 
from Four Studies Using the Continuous Lx)ngitudinal 
Manpower Survey (CLMS) 



12 



Chapter 2 

Exhibit 2. 1 Location of the 16 Study Sites 



24 



18 



ERiC 



JTPA IS-MONTH IMPACTS /LIST OF EXHIBITS* XXI 



Exhibit 2.2 Random Assignment Model for the National JTPA Study 

Exhibit 2.3 Specific Program Services Allowed in Each of the Three 
Service Strategy Definitions 

Exhibit 2.4 Sample Sizes in the 18-Month Study: Full Sample and 
Target Groups, by Service Strategy Subgroup 

Exhibit 2.5 Response Rates in the First Follow-up Survey: 18-Month 

Study Sample, by Target Group and Service Strategy Subgroup 



26 



29 



31 



39 



Chapter 3 

Exhibit 3.1 Key Facts about the 16 Study Sites 

Exhibit 3.2 Selected Population Characteristics of the 16 Study Sites 

Exhibit 3.3 Selected Economic Conditions at the 16 Study Sites 

Exhibit 3.4 Selected Characteristics of Title II-A Terminees at the 
16 Study Sites, Program Years 1987-1989 

Exhibit 3.5 Selected Barriers to Employment Faced by Title II-A 

Terminees at the 16 Study Sites, Program Years 1987-1989 

Exhibit 3.6 Selected Characteristics of JTPA Title II-A Programs at the 
16 Study Sites, Program Years 1987-1989 

Exhibit 3.7 Most Common Service Providers Used by JTPA Title II-A 
Programs at the 16 Study Sites, by Specific Program Service 

Exhibit 3.8 Agency Performance Standards and JTPA Title II-A 

Performance at the 16 Study Sites: Enteied Employment 
Rates of Adult Terminees, Program Year 1988 

Exhibit 3.9 Agency Performance Standards and JTPA Title II-A 

Performance at the 16 Study Sites: Positive Termination 
Rates and Entered Employment Rates of Youth Terminees, 
Program Year 1988 



42 
44 
45 

46 

47 

48 

51 

54 



55 



13 



XXU • JTPA IS-MONTH IMPACTS / LIST OF EXHIBITS 



Exhibit 3.10 Sample Sizes in the 18-Month Study: Full San^iple and 
Target Groups, by Study Site 

Exhibit 3.11 Sample Sizes in the 18-Month Study: Full Sample and 
Target Groups, by Service Strategy Subgroup 

Exhibit 3.12 Treatment Group Enrollment in JTPA: Full Sample and 
Target Groups, by Service Strategy Subgroup 

Exhibit 3.13 Selected Baseline Characteristics of Adults in the Sample: 

Control Group and JTPA Assignees and Enrollees, by Gender 

Exhibit 3.14 Selected Baseline Characteristics of Out-of-School Youths 
in the Sample: Control Group and JTPA Assignees and 
Enrollees, by Gender 

Exhibit 3 . 15 Distribution of the Treatment Group among Subgroups Facing a 
Certain Number of Barriers to Employment: Full Sample and 
Target Groups 

Exhibit 3.16 Service Strategies Reconmiended for the Treatment Group: 
Full Sample and Target Groups 

Exhibit 3.17 Mean Earnings of the Control Group over the 18-Month 
Follow-up Period: Full Sample and Target Groups, by 
Service Strategy Subgroup 

Exhibit 3.18 Receipt of Specific JTPA Services by the Treatment Group: 

Full Sample and Target Groups, by Service Strategy Subgroup 

Exhibit 3.19 Key JTPA Services Received by Treatment Group Members 
Who Were Enrolled in the Program: Target Groups, by 
Service Strategy Subgroup 

Exhibit 3.20 Median Number of Months Enrolled in JTPA among Treatment 
Group Members Who Were Enrolled: Full Sample and Target 
Groups, by Service Strategy Subgroup 

Exhibit 3.21 Treatment Group Enrollment in JTPA in Selected Months 
of ttie 1 8-Month Follow-up Period: Full Sample and Target 
Groups, by Service Strategy Subgroup 



56 



57 



57 



58 



60 



63 



64 



66 



67 



69 



71 



ERIC 



2i) 



JTPA U-MONTH IMPACTS / LIST OF EXHlBfTS • XXIU 



Chapter 4 

Exhibit 4.1 Average Monthly Earnings: Treatment Group and Control Group 78 
Exhibit 4.2 Percentage Enrolled in JTPA Monthly: Treaunent Group 80 

Exhibit 4.3 Impacts on Earnings: Treatment Group 8 1 

Exhibit 4.4 Impacts on Earnings: JTPA Assignees and Enrollees 83 

Exhibit 4.5 Impacts on the Distribution of Total 18-Month Earnings: 

JTPA Assignees 84 

Exhibit 4.6 Impacts on the Percentage Employed and on the Mean Number 

of Weeks and Hours Worked: JTPA Assignees and Enrollees 86 

Exhibit 4.7 Percentage Impacts on Earnings and Its Components: 

JTPA Assignees 86 

Exhibit 4.8 Receipt of Employment and Training Services: Treatment 

Group and Control Group, by Service Strategy Subgroup 93 

Exhibit 4.9 Impacts on Attainment of a Training-Related High School 
Diploma or GED Certificate: JTPA Assignees Overall and 
High School Dropout Subgroup, by Service Strategy Subgroup 99 

Exhibit 4.10 Selected Baseline Characteristics: JTPA Assignees, by 

Service Strategy Subgroup 1 0 1 

Exhibit 4.1 1 Average Monthly Earnings of Adult Women: Treatment 

Group and Control Group, by Service Strategy Subgroup 104 

Exhibit 4.12 Impacts on Earnings: JTPA Assignees and Enrollees, 

by Service Strategy Subgroup 1 06 

Exhibit 4.13 Impacts on the Percentage Employed and on the Mean Number 
of Weeks and Hours Worked: JTPA Assignees, by Service 
Strategy Subgroup 1 09 

Exhibit 4. 14 Percentage Impacts on Earnings and Its Components: 
JTPA Assignees, by Service Strategy Subgroup 



21 



xxrv • jrrPA is-month impacts / list of exhibits 



Exhibit 4.15 
Exhibit 4.16 

Exhibit 4.17 

Chapter S 
Exhibit 5.1 

Exhibit 5.2 
Exhibit 5.3 
Exhibit 5.4 

Exhibit 5.5 

Exhibit 5.6 
Exhibit 5.7 
Exhibit 5.8 

Exhibit 5.9 
Exhibit 5. 10 



Impacts on Total 18-Month Earnings: JTPA Assignees, 

by Selected Key Subgroup 1 16 

Summary of Estimated JTPA Impacts on Earnings and Employment 
over the Full Follow-up Period: Adult Female JTPA Assignees in 
the 18-Month Study Sample, by Service Strategy Subgroup 123 

Estimated Impacts on the Average Quarterly Earnings of Adult 
Women: The JTPA 18-Month Impact Analyses and Previous 
Experhnental Studies 127 



Average Monthly Earnings: Treatment Group and 

Control Group 135 

Percentage Enrolled in JTPA Monthly: Treatment Group 136 

Impacts on Earnings: JTPA Assignees and Enrollees 138 

Impacts on the Distribution of Total 18-Month Earnings: 

JTPA Assignees 140 

Impacts on the Percentage Employed and on the Mean 

Number of Weeks and Hours Worked: JTPA Assignees 

and Enrollees 142 

Percentage Impacts on Earnings and Its Components: 

JTPA Assignees 144 

Receipt of Employment and Training Services: Treatment 

Group and Control Group, by Service Strategy Subgroup 148 

Impacts on Attainment of a Training-Related High School 

Diploma or GED Certificate: JTPA Assignees Overall and 

High School Dropout Subgroup, by Service Strategy Subgroup 155 

Selected Baseline Characteristics: JTPA Assignees, 

by Service Strategy Subgroup 156 

Average Monthly Earnings: Treatment Group and Control 

Group, by Sttrvice Strategy Subgroup 160 



JTPA 18-MONTH IMPACTS / LIST OF EXHIBITS • XX\' 



Exhibit 5.11 
Exhibit 5.12 

Exhibit 5.13 
Exhibit 5.14 
Exhibit 5.15 

Exhibit 5.16 

Chapter 6 
Exhibit 6.1 

Exhibit 6.2 
Exhibit 6.3 

Exhibit 6.4 
Exhibit 6.5 



Impacts on Earnings: JTPA Assignees and Enrollees, 
by Service Strategy Subgroup 

Impacts on the Percentage Employed and on the Mean Number of 
Weeks and Hours Worked: JTPA Assignees, by Service 
Strategy Subgroup 

Percentage Impacts on Earnings and Its Components: 
JTPA Assignees, by Service Strategy Subgroup 



Impacts on Total 18-Month Earnings: JTPA Assignees, 
by Selected Key Subgroup 

Summary of Estimated JTPA Impacts on Earnings and 
Employment over the Full Follow-up Period: Adult Male JTPA 
Assignees in the 18-Month Study Sample, by Service Strategy 
Subgroup 

Estimated Impacts on the Average Quarterly Earnings: 
The JTPA 18-Month Impact Analysis and Previous 
Experimental Studies 



Average Monthly Earnings: Treatment Group and Control 

Group, Overall and by Gender i85 

Impacts on Earnings: JTPA Assignees and Enrollees, by Gender 187 

Impacts on Total 18-Month Earnings: JTPA Assignees 

Ages 16-29, by Gender and Two-Year Age Group 190 

Receipt of Employment and Training Services: 

Female Youth Treatment Group and Control Group, 

by Service Strategy Subgroup 192 

Impacts on Attainment of a Training-Related High 

School Diploma or GED Certificate: Female Youth 

JTPA Assignees Overall and High School Dropout 

Subgroup, by Service Strategy Subgroup 196 



ERIC 



23 



XXVI ♦ JTPA 18-MONTH IMPACTS / LIST OF EXHIBITS 



Exhibit 6,6 Selected Baseline Characteristics: Female Youth JTPA 

Assignees, by Service Strategy Tubgroup 198 

Exhibit 6.7 Impacts on Earnings: Female Youth JTPA Assignees 

and Enrollees, by Service Strategy Subgroup 201 

Exhibit 6.8 Percentage Impacts cn Earnings and Its Components: 

Female Youth JTPA Assignees, by Service Strategy Subgroup 202 

Exhibit 6.9 Receipt of Employm^'nt and Traming Services: 

Male Youth Treatme it Group and Control Group, 

by Service Strategy Subgroup 206 

Exhibit 6. 10 Impacts on Attainment of a Training-Related High School Diploma 
or GED Certificate: Male Youth JTPA Assignees Overall and 
High School Dropout Subgroup, by Service Strategy Subgroup 209 

Exhibit 6. 1 1 Selected Baseline Characteristics: Male Youth JTPA 

Assignees, by Service Strategy Subgroup 210 

Exhibit 6. 12 Impacts on Earnings: Male Youth JTPA Assignees and 

Enrollees, by Service Strategy Subgroup 213 

Exhibit 6. 13 Percentage Impacts on Earnings and Its Components: Male 

Youth JTPA Assignees, by Service Strategy Subgroup 215 

Exhibit 6. 14 Impa^.^ on Total 18-Month Earnings: Female Youth JTPA 

Assignees, by Selected Key Subgroup 218 

Exhibit 6. 15 Impacts on Total 18-Month Earnings: Male Youth JTPA 

Assignees, by Selected Key Subgroup 222 

Exhibit 6.16 Summary of Estimated JTPA Impacts on Earnings and 

Employment over the Full Follow-up Period: Out-of-School 

Youth JTPA Assignees in the 18-Month Study Sample, 

by Gender and Service Strategy Subgroup 227 

Exhibit 6. 17 Estimated Impacts on the Average Quarterly Earnings of 

Out-of-School Youths: The JTPA 18-Month Impact Analysis 

and Previous Experimental Studies 231 



24 



ERIC 



JTPA IS-MONTH IMPACTS / LIST OF EXHIBITS • XX\'U 



Chapter 7 
Exhibit 7.1 

Exhibit 7.2 

Exhibit 7.3 

Exhibit 7.4 

Exhibit 7.5 

Exhibit 7.6 
Exhibit 7.7 

Exhibit 7.8 
Exhibit 7.9 
Exhibit 7.10 
Exhibit 7.11 
Exhibit 7.12 



Impacts on Total 18-Month Earnings and Employment: JTPA 
Assignees, by Target Group 

Service Strategies Reconunended: Treatment Group, 
by Target Group 

Receipt of Specific JTPA Services: Treatment Group, by Target 
Group and Service Strategy Subgroup 

Difference in the Mean Number of Hours of Employment and 
Training Services Received by the Treatment Group and Control 
Group: Target Groups, by Service Strategy Subgroup 

Attainment of a Training-Related High School Diploma or 
GED Certificate: Treatment Group and Control Group, 
by Target Group and Service Strategy Subgroup 

Impacts on Quarterly and 18-Month Earnings: JTPA Assignees, 
by Target Group and Service Strategj' Subgroup 

Percentage Impacts on Total 18-Month Earnings and Selected 
Earnings Components: JTPA Assignees, by Target Group and 
Service Strategy Subgroup 

Impacts on Total 18-Month Earnings: JTPA Assignees, by 
Target Group and Selected Key Subgroup 

Impacts on Total 18-Month Earnings: Adult and Out-of-Schoc? 
Youth JTPA Assignees, by Study Site 

impacts on Total 18-Month Earnings: JTPA Assignees, 
by Target Group and Study Site 

Correlation of Impacts on Total 18-Month Earnings across 
Target Groups, b> Study Site 



238 



241 



242 



245 



247 



250 



254 



258 



261 



263 



264 



Influence of Selected Site and Baseline Characteristics, and Sets of 
Characteristics, on the Size of the Impact on Total 18-Month 
Earnings: Adult and Out-of-School Youth JTPA Assignees 272 



XXVUl • JTPA J8-MONTH IMPACTS / LIST OF EXHIBITS 



Appendix A 
Exhibit A.l 



Exhibit A.2 



Appendix B 
Exhibit B.l 



Exhibit B.2 



Exhibit B.3 



Exhibit B.4 



Exhibit B.5 



Exhibit B.6 



Exhibit B.7 



Exhibit B.8 



Results of the Multivariate Discriminant Analysis of BaseJi.ne 
Characteristics of the Treatment Group and Control Group; 
The 18-Month Study Sample, by Target Group 

Baseline Characteristics Used as Regressors in the 
Multivariate Analysis of Treatment-Control Group Differences 



Selected Baseline Characteristics of Adult Women: 
JTPA EnroIIees in the 18-Month Study Sample and 
the National ComparUon Samples of JTPA EnroUees 
and Terminees 



283 



284 



296 



Selected Baseline Characteristics of Adult Men: JTPA EnroIIees 

in the 18-Month Study Sample and the National Compar'-on 

Samples of JTPA EnroIIees and Terminees 297 

Selected Baseline Characteristics of Out-of-SchooI Female Youths; 
JTPA EnroIIees in the 18-Month Study Sample and tiie National 
Comparison Samples of JTPA EnroIIees and Terminees 298 



Selected Baseline Characteristics of Out-of-School Male Youths: 
JTPA EnroIIees in the 18-Month Study Sample and the National 
Comparison Samples of JTPA EnroIIees and Terminees 

In-Program Experiences of Adult Women: JTPA EnroIIees in the 
18-Month Study Sample and the National Comparison Sample of 
JTPA Terminees 

In-Program Experiences of Adult Men: JTPA EnroIIees in the 
18-Month Study Sample and the National Comparison Sample 
of JTPA Terminees 

In-Program Experiences of Out-of-School Female Youths: 
JTPA EnroUees in the 18-Month Study Sample and the 
National Comparison Sample of JTPA Terminees 

In-Program Experiences of Out-of-School Male Youths: 
JTPA EnroUees in the 18-Month Study Sample and the 
National Comparison Sample of JTPA Terminees 



299 



300 



301 



302 



303 



JTPA IS-MONTH IMPACTS / LIST OF EXHIBITS • XXIX 



Appendix C 

Exhibit CI 
Exhibit C.2 

Appendix D 
Exhibit D.l 

Exhibit D.2 

Exhibit D,3 

Exhibit D.4 

Appendix £ 
Exhibit E.l 

Exhibit E.2 

Exhibit E.3 

Exhibit E.4 



Imputation of Missing Earnings and Employment Variables 

Coverage Rates for Eammgs Data from State UI Agencies, 
by Quarter, relative to Random Assignment 



Baseline Chvacteristics Used as Regressors in Adjusting 
for Differences Between the Treatment Group and 
Control Group, by Target Group 

Derivation of Estimated Percentage Impacts on the 
Number of Weeks and Hours Worked, Adjusted for 
Survey Nonresponse Bias 

Estimated Impacts on Earnings, with and without 
Nonrespondent Imputations: Adult Women 

Estimated Impacts on Employment, with and without 
Nonrespondent ImpuUtions: Adult Women 



Distribution of Survey- and UI-Based Estimates of Mean 
12-Month Earnings: The 12-Month Comparison Sample, 
by Target Group 

Survey- and UI-Based Estimates of Mean 12-Month Earnings: 
The 12-Month Comparison Sample, by Target Group and 
Study Site 

Survey- and UI-Based Estimates of Mean 12-Month Earnings 
in Sites Located Near a State Border and Those Not: 
The 12-Month Comparison Sample, by Target Group 

Survey- and UI-Based Estimates of Mean Quarterly and 
12-Month Earnings: The 12-Month Earnings: The 12-Month 
Comparison Sample, by Target Group 



317 



318 



338 



341 



342 



343 



357 



358 



359 



360 



ERIC 



£7 



XXX • JTPA 18-MONTH IMPACTS / LIST OF EXHIBITS 



Exhibit E.5 Survey- and UI-Based Estimates of the Percentage Employed; 
The 12-Month Comparison Sample, by Target Group 

Exhibit E.6 UI-Based Estimates of the Percentage Employed: The 12-Month 
Comparison Sample and the Subsample with Six Quarters of UI 
Earnings Data, by Target Group 

Exhibit E.7 Survey and UI-Based Estimates of Mean Quarterly and 
12-Month Earnings per Employed Sample Member: 
The 12-Month Comparison Sample, by Target Group 



Exhibit E.8 Percentage of Person-Quarters in which Employment 
Was Reported in the First Follow-up Survey Only, 
UI Earnings Data Only, Both Data Sources, or Neither 
Sources: The 12-Month Comparison Sample, 
by Target Group 

Exhibit E.9 Mean 12-Month Earnings for Person-Quarters with Employment 
Reported in One Data Source or BotI . Sources: The 12-Month 
Comparison Sample, by Target Group 

Exhibit E.IO Survey- and UI-Based Estimates of Program Impacts on 
Earnings: JTPA Assignees in the 12-Month Comparison 
Sample, by Target Group 



Appendix F 

Exhibit F. 1 Distribution of Further JTPA Aaivity and Service Receipt after 
Random Assignment: A Random Sample of Treatment Group 
Nonenrollees in the 18-Month Study Sample 



Appendix G 

Exhibit G. 1 Estimated Impacts on Latent Hourly Wage Rates in the Fifth 

and Sixth Quarters after Random Assignment: JTPA Assignees 
in the 18-Month Study Sample, by Target Group and Service 
Strategy Subgroup 



JTPA 18-MONTH IMPACTS / LIST OF EXHIBfTS • XXXI 



Appendix H 
Exhibit H.1 

Exhibit H.2 

Exhibit H.3 

Exhibit H.4 
Exhibit H.5 

Exhibit H.6 

Exhibit H.7 
Exhibit H.8 
Exhibit H. 9 
Exhibit H. 10 
Exhibit H. 11 

Exhibit H. 12 



Percentage Enrolled in JTPA Monthly: Female Youth 
Treatment Group 

Impacts on the Distribution of Total 18-Month Earnings: 
Female Youth JTPA Assignees 

Impacts on the Percentage Employed and on the Mean Number 
of Weeks and Hours Worked: Female Youth JTPA Assignees 
and Enrollees 

Percentage impacts on Earnings and Its Components: 
Female Youth JTPA Assignees 

Average Monthly Earnings of Female Out-of-School 
Youths: Treatment Group and Control Group, by 
Service Strategy Subgroup 

Impacts on the Percentage Employed and on the Mean Number 
of Weeks and Hours Worked: Female Youth JTPA Assignees, 
by Service Strategy Subgroup 

Percentage Enrolled in JTPA Monthly: Male Youth 
Treatment Group 

Impacts on the Distribution of Total 18-Month Earnings: 
Male Youth JTPA Assignees 

Impacts on the Percentage Employed and on the Mean Number 
of Hours Worked: Male Youth JTPA Assignees and Enrollees 

Percentage Impacts on Earnings and Its Components: 
Male Youth JTPA Assignees 

Average Monthly Earnings of Male Out-of-School 
Youths: Treatment Group and Control Group, 
by Service Strategy Subgroup 

Impacts on the Percentage Employed and on the Mean Number 
of Weeks and Hours Worked: Male Youth JTPA Assignees, 
bv Service Stratesv Suheroun 



378 
378 

379 
380 

381 

382 
383 
383 
384 
385 

386 

387 



29 

erJc 



XXXU • JTPA 18-MONTH IMPACTS / LIST OF EXHIBITS 



Exhibit H. 13 Percentage Enrolled in JTPA Montnly: Treatment Group, 

All Youths 388 

Exhibit H. 14 Impacts on Earnings: All Youth JTPA Assignees and Enrollees 388 

Exhibit H.15 Impacts on the Distribution of Total 18-Month Earnings: 

All Youth JTPA Assignees 389 

Exhibit H. 16 Impacts on the Percentage Employed and on the 
Mean Number of Weeks and Hours Worked: 

All Youth JTPA Assignees and Enrollees 390 

Exhibit H.17 Percentage Impacts on Earnings and Its Components: 

All Youth JTPA Assignees 391 

Exhibit H. 18 Impacts on Attainment of a Training-Related High School 
Diploma or GED Certificate: All Youth JTPA Assignees 
Overall and High School Dropout Subgroup, by Service 
Strategy Subgroup 392 

Exhibit H. 19 Impacts on Earnings: All Youth JTPA Assignees and 

Enrollees, by Service Strategy Subgroup 393 

Exhibit H.20 Impacts on the Percentage Employed and on the Me.an Number 
of Weeks and Hours Worked: All Youth JTPA Assignees, 
by Service Strategy Subgroup 394 

Exhibit H.21 Impacts on Total 18-Month Earnings: All Youth JTPA 

Assignees, by Gender and Ethnicity 395 

Exhibit H.22 Deviations of Impacts on the Total 18-Month Earnings of Each 
Two-Year Youth Cohort from the Trend in Impacts on the 
18-Month Earnings of Young Adults Ages 22 to 29: JTPA 
Assignees, by Gender and Service Strategy Subgroup 395 

Exhibit H.23 Impacts on the Number of Months of Continuous 

Nonemployment between Random Assigmnent and the 

First Job after Random Assignment: Youth Treatment 

Group and Control Group, by Gender and Service 

Strategy Subgroup 396 



30 



ERIC 



Executive Summary 



THE National JTPA Study was commissioned by the Employment and Training 
Administration of the U.S. Department of Labor (DOL) in 1986 to measure the 
impacts and costs of selected employment and training programs funded under Title II-A 
of the Job Training Partnership Act of 1982, which is targeted to serve economically 
disadvantaged Americans. This report presents interim estimates of program impacts 
on the earnings and employment of adults and out-of-school youths in 16 local service 
delivery areas during the first 18 months after their acceptance into the program. 

Estimates of longer term program impacts on earnings, employment, and welfare 
benefits, and an analysis of program costs and benefits, will appear in the final report of 
the study (forthcoming, from Abt Associates Inc.). A companion repon on the study's 
implementation (Doolittle, forthcoming) describes the JTPA programs operated in the 
study sites and the types of JTPA-funded services provided to members of the study 
sample. 

The National JTPA Study 

This study grew out of the recommendations of the Job Training Longitudinal Survey 
Advisory Panel, a group of nationally recognized experts in employment nd training 
research formed to advise DOL on the evaluation of JTPA (Stromsdorfer et al., 1985). 
After reviewing evaluations of Comprehensive Employment and Training Act (CETA) 
programs, the panel concluded that the only reliable way to measure the impacts of 
employment and training programs was to conduct a classical experiment, in which 
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program applicants are randomly assigned either to a treatment group, which is allowed 
access to tlie program, or to a control group, which is not. Random assignment assures 
that the two groups do not differ systematically in any way except access to the program. 
Thus, subject only to the uncertainty associated with sampling error, any subsequent 
differences in outcomes between the two groups can confidently be attributed to the 
program. These differences are termed program impacts. 

Although random assignment designs have been used to evaluate a number of 
demonstration projects and state programs, the Employment and Training Administration 
was the first federal agency to apply this approach to an ongoing national program. 
Because of its rigorous design, the National JTPA Study provides the first reliable 
estimates of the impacts of the largest employment and training program sponsored by 
the federal government. 

In the National JTPA Study 20,601 JTPA applicants in 16 service delivery areas 
(SDAs) across the country were randomly assigned to the treatment group or the control 
group over the period November 1987 through September 1989. The earnings and 
employment outcomes of both groups were then measured through follow-up surveys and 
administrative records obtained from state unemployment insurance agencies. Data on 
the baseline characteristics of the two groups were collected as part of the program intake 
process, and information about the employment and training services received was 
obtained from follow-up surveys and SDA records. 

The study sites were not chosen to be representative of the nation in a statistical sense, 
but they do reflect the diversity of local programs and local environments in JTPA. In 
particular, the performance of the sites during the study period, as measured by JTPA 
performance indicators, was not noticeably different from that of all SDAs nationally.^ 

The 18-Month Impact Analysis 

This report provides estimates of the impact of JTPA Title II-A on the earnings and 
employment of four target groups-^z&nXi women and men (ages 22 and older) and female 
and male out-of-school youths (ages 16 to 21)-— over the first 18 months after random 
assignment. Adult women make up 30 percent of the national JTPA population; adult 
men, 2' percent; and out-of-school youths, 23 percent. In-school youths, who are not 
included in this study, form the remaining 22 percent. 



1 . See Appendix B and Chapter 3 for comparisons of the 16 study sites with all SDAs nationally. 
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The analysis is based on a subsample of 17,026 sample members whose First Follow- 
up Survey intexview was scheduled at least 18 months after random assignment.- For each 
target group we estimated impacts for a number of different subgroups, defined by the types 
of program services recommended for them and by their baseline characteristics. 

Becaus:^the study was designed to measure the effects of JTPA as it normally operates, 
the analysis investigates which JTPA-funded services were working well for those 
recommended to receive them; the analysis does not assess possible alternatives to the 
existing program. By identifying those groups for whom Title II-A is having positive 
effects and those for whom it is having no effect— or even a negative effect— we hope to 
help policymakers in their efforts to identify those parts of the program that need 
improvement. This analysis cannot, however, tell policymakers how to improve the 
program, since it does not compare alternative programs for similar people. Rather, it 
measures only the effects of the existing program on the people it actually served over the 
study period. 

In the remainder of this Executive Summary we first provide an overview of the 
estimated effects of the program on the earnings and employment of the four main target 
groups— adult women and men, and female and male youths. We then present more 
detailed findings for adult and youth subgroups, in turn, and conclude with implications 
of the findings for the JTPA program and future research. 

Overall Impacts on Earnings and Employment, by Target Group 

JTPA Title II-A had generally positive effects on the earnings and employment of adults 
in the study sites . As shown in the top panel of Exhibit S . 1 , access to the program increased 
the average 18-month earnings of the adult women randomly assigned to the treatment 
group ("JTPA assignees") by an estimated $539, or 7.2 percent of the control group mean. 
Access to the program also increased the percentage of women employed at some time 
during the follow-up period by 2.1 percentage points. Because these estimates are 
statistically significant (as indicated by the asterisks beside them), we take them to be 
reliable evidence of positive impacts on earnings. In this analysis we accept only statisti- 
cally significant estimates as evidence of real program effects. 

The estimated program impacts for adult men- an earnings g'iin of $5.50, or 4.5 
percent, and an increase in the percentage employed of 2.8 percentage points— were 
similar in size to those for adult women, but the estimated impact Oi) earnings was not 
statistically significam. 

2. Within this 18-monlh study sample, First Follow-up Survey data are available for 14,442 sample 
members, or 84.8 percent of the sample. 
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Exhibit S.J Impacts on Total J 8-Month Earnings and Employment: JTFA Assignees 
and Enrollees, by Target Croup 
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In contrast to the findings for adults, the program had little or no effect on the average 
earnings of female youths (a statistically insignificant earnings loss of $-182, or -2.9 
percent), and the program actually reduced the earnings of male youths, on average— as 
evidenced by a large, statistically significant loss of $-854, or -7.9 percent, ever the 18- 
month period. Access to JTPA had no significant effect on the 18-month employment rates 
of either female or male youths. 

Hence, the findings for the female youths are clear-cut: JTPA had virtually no effect 
ontheir earnings or employment. Butthefindings for male youths are less clear. Asshown 
later in this summary, almost all of the negative average impact on the earnings of male 
youths is concentrated among those who reported having been arrested between age 16 and 
random assignment (25 percent of the male youth treatment group). ^ Thus, the estimated 
impact for most male youths (the 75 percent with no previous arrest) was negligible. 

The estimates discussed above are average impacts on the earnings and employment 
of all sample members assignee to the treatment group. Although all of these assignees 



3. Furthermore, as noted later in this summary, there is some question about the large» negative impact 
estimated for male youths v^ith a previous arrest. 
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were ^iven access to JTPA, not all of them actually enrolled in the program. The bonom 
panel of Exhibit S,l presents our best estimates of program impacts on the earnings and 
employment of JTPA enroUees (assignees who were later enrolled)/ 

Estimated impacts per enrollee— both gains and losses— were about 60 percent to 70 
percent larger than impacts per assignee, depending on the target group. The estimated 
earnings gains of adult women and men who were enrolled in JTPA were $873 and $935, 
respectively. Impacts on youths were earnings losses of $-294 for females and $-1 ,356 for 
mal es The impact of the program on the percentage of enrollees in each target group who 
were employed ranged from an increase of 2.4 percentage points for male youths to an 
increase of 4.8 percentage points for adult men. 

It is important to understand that the impact per assignee and the impact per enroUee 
are not two different estimates of the overall effect of the program. They simply spread 
the total estimated program effect on the sample over a larger group (assignees) or a smaller 
group (enrollees). Thus, the two sets of estimates are entirely consistent; they just measure 
different concepts. In the remainder of this Executive Summary, we focus on the estimated 
impacts per assignee, because they are the most reliable, direct experimental evidence of 
the effects of the program. 

Impacts on earnings reflect program effects on both the amount of time treatment group 
members worked and how much they were paid per hour worked. Exhibit S,2 shows 
estimated impacts on the average number of hours worked by assignees and average 
earnings per hour worked over the follow-up period, expressed as percentages of the 
corresponding control group means. The percentage impacts on these two components of 
earnings approximately sum to the percentage impact on total earnings per assignee/ 



4 . To derive eslimaies for enrollees, ii was necessary to assume that there was no impact on the earnings 
and employment of nonenroUees. There is evidence, however, that about half of all nonenrollees had some 
contact with the program after random assignment and received some— usually minimal— program services . As 
a result, the estimates in the bottom panel probably overstate somewhat the true impact on enrollees, 'vhilc the 
estimated impacts per assignee undersute the true impact on enrollees. Thus, the true impact on enrollees 
probably lies somewhere between these two estimates. The estimates for enrollees also adjust for the fact lliai 
3 percent of the conu-ol group became enrolled in JTPA, despite the experiment' s embargo on their panicipation 

5. As was true of the estimated impact per assignee for male youths, the large, negative impact per 
enrollee for male youths is due almost entirely to a very large estimated impact for those male youth enrollees 
v^ith a previous arrest. 

6. Because the impacts on earnings per hour worked were estimated indirectly, we did not calculate 
significance levels for these impacts. 
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Exhibit S.2 Percentage Impacts on Total ] 8-Month Earnings and Its Components: 
JTPA Assignees and Enrollees, by Target Group 





Adult 


Adult 


Female 


Male 


Percentage 


women 


men 


youths 


youths 


impact on: 


0) 


a) 


(3) 


(4) 


Earnings per assignee 


7.2%*** 


4.5% 


-2.9% 


-7.9%** 


Hours worked per assignee 


3.7 


4.5** 


-4,7 


-6.8*» 


Earnings per hour worked 


3.4* 


0.0* 


1.9* 


-1.2* 


Sample size 


6,474 


4,419 


2,300 


1,748 



a. Tc«u of aUtitlical tipuf»c«ncc were net performed for irapacU oo eanunft per hour wocked. 



• Suiiniciny f i{nific«Dl «l the .1 level, tithe .05 level ••• ai Ibc .01 level (two-uUed lesl). 

As shown in Exhibit S.2, the 7.2 percent increase in earnings for adult women reflects 
a combination of a 3.7 percent increase in hours worked and a 3.4 percent increase in 
average hourly earnings among those who worked. The earnings gain for adult men, on 
the other hand, was entirely attributable to a 4.5 percent rise in hours worked, with no 
increz * 'n hourly earnings. 

Among female youths a -4.7 percent reduction in hours worked mor3 than offset a 
1 .9 percent increase in hourly earnings to produce the negligible impact on total earnings 
that we saw earlier. Among male youths the -7.9 percent loss in total earnings was 
primarily attributable to a decrease in hours worked (of -6.8 percent). 

Overall, then, JTPA appears to have had modest positive effects on the earnings and 
employment of adult women and men. But the program appears to have had virtually no 
effect on the earnings and employment of female youths and most male youths . In contrast, 
it may have had a large, negative impact on the earnings of those male youths who had 
been arrested before they applied to JTPA. 

When estimated separately by site, positive but generally insignificant earnings effects 
were obtained in most sites for adult women and adult men, negative but generally 
insignificant earnings effects were obtained for male youths, and a majority of sites yielded 
negative but insignificant earnings effects for female youths (not shown herej. Thus, the 
main 18-month earnings findings by target group were found to be widespread across the 
16 SDAs in the study. And despite wide variation in the magnitude of these estimated 
effects, the sites did not differ significantly from one another in the degree to which JTPA 
affected earnings in any individual target group. 

In an attempt to explain the variation in impact estimates across sites, we conducted 
a limited exploratory analysis of local factors that might influence program impacts. Three 
types of factors were considered: (1) characteristics of the JTPA programs; (2) prevailing 
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labor market conditions; and (3) the types of persons accepted into the programs. But no 
clear patterns emerged from the analysis; and almost all of the findings were statistically 
insignificant, which is probably due to the small samples at each site and the limited number 
of sites involved. 

Findings for Subgroups of Adult Women and Men 

The impacts presented in the previous section are estimates of the average effects of the 
program on each target group in the study sample. Because JTPA provides a number of 
different employment and training services to a wide range of program applicants, it is 
important to analyze how program impacts varied with the types of services offered and 
the characteristics of the applicants. In this section we therefore present estimates of 
program impacts on the earnings of adult subgroups defined by the services that program 
intake staff recommended for them and by selected personal characteristics. 



Servic- Strategies Recommended 

For purposes of this analysis, members of the study sample were classified into three 
service strategy subgroups based on the services that program intake staff recommended 
fo" each sample member prior to random assignment.*' Applicants recommended for 
classroom training in occupational skills were placed in the classroom training subgroup. 
Those recommended for on-the-job training (OJT) were placed in the OJT/JSA subgroup 
(so named because many of the treatment group members in this subgroup were enrolled 
in job search assistance while searching for either an on-the-job training position or an 
unsubsidized job). Because JTPA staff sometimes recommend combinations and se- 
quences of services, applicants placed in either of these subgroups may also have been 
recommended for any of several other services, including job search assistance, basic 
education, work experience, or miscellaneous other services. Those applicants recom- 
mended for one or more of these services— but neither classroom training in occupational 
skills nor on-the-job training— were placed in the third subgroup: other services.* 



7. Service strategy subgroups were defined based on the services rccoiiunended rather than the services 
received for t>vo reasons. First, it was not possible to identify control group members who were comparable 
10 the treaiment group members ^o received particular JTPA services, whereas it Mfas possible to identify 
control group members who were recommended for the same services as treatment group members. Second, 
and more fundamentally, since program staff can recommend services but cannot ensure that applicants 
participate in those services, recommended services represent the operative program decision to be evaluated 

8. A few applicants designated for this other service subgroup were recommended for classroom traming 
m occupational skills or on-the-job training as pan of ''customized training.** 
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Exhibit S.3 Service Strategies Recommended: 
Adult JTPA Assignees, by Gender 



women 



Adult 



Adult 
men 



Service strategy 



(1) 



(2) 



Classroom training 



44.0% 



24.6% 



OJT/JSA 



35.0 



48.7 



Other services 
Sample size 



21.0 
4,465 



26.7 
3,759 



As shown in Exhibit S.3, nearly half of all adult men in the treatment group were 
recommended for the OJT/JSA service strategy, with the remainder about equally divided 
between the classroom training and other services strategies. Women were more likely 
than men to be recommended for classroom training (44 percent versus 25 percent) and 
less likely to be recommended for OJT/JSA (35 percent versus 49 percent). 

It is important to note that program intake staff recommended services based on the 
individual applicants' employment needs and qualifications, as well as their personal 
preferences. The service strategy subgroups therefore differed from one another not only 
in terms of the service recommendations but also in terms of personal characteristics. 

Enrollment Rates and Duration, by Service Strategy Subgroup 

After assessment and recommendation of services, two-thirds of the applicants accepted 
by intake staff were randomly assigned to the treatment group, which was allowed access 
to JTPA, and one-third were assigned to the control group, which was excluded from JTPA 
for 18 months.' 

As noted above, not all treatment group members would ultimately become enrolled 
in JTPA. Enrollment rates differed by service strategy subgroup, but overall they were 
quite similar for adult women and men. Within the treatment group as a whole. 65 percent 
of adult women and 61 percent of adult men were enrolled in JTPA at some time during 
the 18-month foPow-up period. Enrollment rates were highest in the classroom training 
subgroup (73 percent and 71 percent for adult women and men, respectively) and lowest 
in the OJT/JSA subgroup (55 percent and 57 percciu). 

9, This embargo on services to control group members was successfully implemented. Over the course of 
the 18-month follow-up period, only 3 percent of control group members became enrolled in JTPA. 



,^8 
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The duration of enrollment in the program also differed by service strategy, ranging 
from a median length of about 2 months for women and men in the OJT/JSA and other 
services subgroups to median lengths of enrollment of 4 to 6 months in the classroom 
training subgroup. Generally, there was little difference by gender in the duration of 
enrollment except that women in classroom training tended to stay in the program about 
two months longer than men. 

Services Received, by Service Strategy Subgroup 

Within the classroom training subgroup the most common JTPA services received by 
treatment group members who became enrolled in the program were classroom training 
in occupational skills, basic education, and job search assistance. EnroUees in the 
OJT/JSA subgroup were most likely to receive on-the-job training or job search assistance, 
or both. In the other services subgroup the most common services adults received were 
job search assistance and miscellaneous services, such as job-readiness training. Exhibit 
S.4 shows that between 82 percent and 89 percent of the enrollees in each service strategy 
subgroup received one or both of the two key services characteristic of that service 
strategy. Thus, the three service strategy definitions represent distinctly different mixes 
of services actually received, as well as services recommended. 

The impacts of the program do not depend solely, however, on the JTPA services 
received by those in the treatment group. Instead, the impacts reflect the difference 
between the services received by those given access to JTPA and the services they would 
have received if they had been excluded from the program. That is, the benchmark against 
which we measure the effects of JTPA is the services available elsewhere in the 
community, not a total absence of services. Our measure of the services the treatment 
group would have received if they had been excluded from the program is those received 
by the control group, who were excluded from the program. 

Since we measure impacts per assignee (treatment group member), the relevant 
comparison is in terms of services per assignee, including those who were never enrolled 
in JTPA. As expected, the largest treatment-control group difference in the classroom 
/ra/;2/n^^wZ>^rowpwas in receipt of classroom training inoccupational skills. Among adult 
women 49 percent of the treatment group received this service, whereas only 29 percent 
of the control group did. Among adult men these figures were 40 percent versus 24 
percent. 

Adult treatment group members in the OJT/JSA subgroup were much more likely than 
control group members to receive on-the-job training. We estimate that 29 percent of the 
women and 27 percent of the men in the treatment group in this subgroup received OJT, 



XLU • JTPA 18-MONTH IMPACTS / EXECUTIVE SUMMARY 



Exhibit SA Key JTPA Services Received by Treatment Group 

Members Who Were Enrolled in the Program: Adults, by 
Gender and Service Strategy Subgroup 

% of enrollees receiving 
one or both services 



Key services 

in service strategy subgroup 



Adult 
women 
(1) 



Adult 
men 

(2) 



Classroom training in occupational 
skills/basic education"* 



Classroom training subgroup 
88.8% 85.5% 



On-the-job training/ 
job search assistance 



Job search assistance/ - 
miscellaneous* 

Sample size 



OJT/JSA subgroup 

87.8% 86.5% 

Other services subgroup 



82.3% 
2,883 



88.7% 
2,286 



a. 'Basic cducaiion* includes Adult Basic Education (ABE), high school or General Educational 
Development (GED) preparation, and English as a Second Language (ESL). 

b, "Miscellaneous* includes assessment, job-readiness training, customized training, vocational exploration, 
job shadowing, and iryout employment, among other services. 



whereas less than 1 percent of the corresponding control groups received this service, since 
OJT is typically not funded by non-JTPA providers. We were not able to measure control 
group receipt of job search assistance from non-JTPA providers, and so we could not 
estimate the treatment-control group difference for that service. 

As noted earlier, the most common JTPA services provided to adults in the other 
services subgroup were job search assistance and miscellaneous services. Around 25 
percent of adult treatment group members in this subgroup received the former service, and 
about 30 percent received the latter. We were unable to measure receipt of these services 
from non-JTPA providers and therefore cannot estimate the treatment-control group 
difference. 

JTPA thus represented a clear increment in the services available elsewhere in the 
community, at least in the classroom training and OJT/JSA subgroups where we could 
measure the treatment-control group differential, but that increment was relatively modest. 
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Among adult women and men the average assignee in the classroom training subgroup 
received only an additional 95 to 1 10 hours of classroom uaining in occupational skills, 
and the average assignee in the OJT/JSA subgroup likewise received only an additional 
104 to 114 hours of on-the-job training. 



Impacts on Educational Attainment, by Service Strategy Subgroup 

An intermediate effect of the increment in services received by treatment group members 
was an increase in educational attainment among those who were high school dropouts. 
Dropouts made up around 30 percent of the aduh target groups. Our analysis focuses on 
the attainment of a training-related high school credential, which we define as both having 
received a school or training service am/ having received a high school diploma or General 
Educational Development (GED) certificate at some time duruig the 18-month follow-up 
period. 

As might be expected, the increase in educational attainment was greatest among those 
dropouts recommended for the classroom training service strategy. Exhibit S.5 indicates 
that nearly 30 percent of the adult dropouts in the classroom training treatment group 
received a training-related high school credential, whereas only 1 1 percent of the control 
group did— for impacts that were highly significant in the cases of both genders. There 
were smaller, but still statistically significant, increases in the proportions of female 
dropouts in the other services subgroup and male dropouts in the OJT/JSA subgroup who 
attained a high school credential as a result of the program. But there were no significant 
effects on educational attainment among women in the OJT/JSA subgroup or men in the 
other services subgroup. 



Exhibit S.S Impacts on Attainment of a Training-Related High School Diploma or GED 

Certificate: Adult JTPA Assignees Who Were High School Dropouts, by Gender 



Service 
strategy 
subgroup 




Adult women 






Adult men 




% attaining 


HS/GED 


Impaci, in 
% points 
(3) 


% attaining 


HS/GED 


Impact, in 
% points 
(6) 


Assignees 
(1) 


Controls 
(2) 


Assignees 
(4) 


Controls 
(5) 


Classroom 












16.0*** 


training 


29,2% 


11.3% 


17. 9*** 


21.3% 


11.3% 


OJT/JSA 


9.1 


10.9 


-1.8 


8.4 


4.4 


4.0** 


Other services 


17.4 


9.8 


7.6** 


10.2 


8.7 


1.5 


All subgroups 


19.1 


10.8 


8.2*** 


12.7 


6.7 


6.0*** 


Sample size 






7,575 






1,258 



a Assignees »nd conlro. group members who were high school dropouts. 

♦ Suiislictlly sigmficjini «i the .10 IcvcU « the .05 level, ii the .01 leveJ (two-uilcd tea). 
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Impacts on EARhONcs, by Service Strategy Subgroup 

Exhibit S. 6 shows the estimated program impacts on the earnings of adult women and men 
in each service strategy subgroup. As shown in the second column of the top panel of the 
exhibit, impacts on the earnings of adult women in the classroom training subgroup 
followed the expected pattern for this type of service: an earnings loss in the first quaner, 
representing an initial investment of time in training, followed by a payback period of 
rising earnings gains in the next five quarters, with statistically significant gains of $1 44 
and $188 in the last two quarters of the follow-up period. The overall 18-month earnings 
gain of $398 for women in this subgroup was not statistically significant. This gain 
reflected an estimated 8.9 percent program-induced increase in the hourly earnings of 
those women who worked, which more than offset an insignificant -2.5 percent drop in 
the average number of total hours worked by all adult women over the follow-up period 
{estimates not shown in the exhibit). 

The estimated impacts on the earnings of adult men in the classroom training subgroup 
are less clear. None of the impacts on quarterly earnings was significantly different from 
zero, nor was the overall impact on total earnings over the follow-up period. Moreover, 
the program had no significant impact on the employment rate or hours of work over the 
follow-up period for this subgroup of men (estimates not shown). Thus, there is no 
evidence of a program impact on the earnings and employment of this subgroup. 

In contrast to the pattern for women in the classroom training subgroup, women in the 
OJT/JSA subgroup (middle panel of the exhibit) experienced an immediate and sustained 
positive impact on average earnings throughout the follow-up period, as might be expected 
with a strategy that emphasizes immediate placement in either an on-the-job training 
position or a regular job. Women in the OJT/JSA subgroup had significant quarterly 
earnings impacts of $ 109 to $ 144 in five of the six quaners, with a significant gain of S742 
over the follow-up period as a whole. 

Men in the OJT/JSA subgroup experienced estimated gains of similar magnitude in 
five of the six quaners and over the follow-up period as a whole, although the estimated 
impacts were less often statistically significant. Over the 18 months men in this subgroup 
experienced significant earnings gains of $781. 

Both women and men in the OJT/JSA subgroup experienced a positive and significant 
impact on hours worked; and men, on their employment rate (estimates not shown in the 
exhibit). Indeed, the earnings gains of both women and men in this subgroup were due 
primarily to increases in the number of hours worked per sample member, rather than to 
higher hourly earnings while employed. 
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Exhibit S.6 Impacts on Quarterly and 18-Month Earnings: Adult JTPA 
Assignees, by Gender and Service Strategy Subgroup 





Adult women 


Adult men 




Ccntrol 


Impact per 


Control 


Impact per 




mean 


assignee 


mean 


assignee 


Period 


0) 


(2) 


(3) 


(4) 






Classroom training subgroup 




Quarter 1 


S 714 


$ -70* 


$ 1,440 


$ -101 


2 


938 


5 


1,714 


126 


3 


1,066 


52 


1,884 


213 


4 


1,189 


79 


2,184 


50 


5 


1,253 


144** 


2,171 


151 


6 


1,230 


188*** 


2,387 


- 21 


All quarters 


6,391 


398 


11.780 


418 


Sample size " 




2M7 




lfi57 






OJT/JSA 


subgroup 




Quarter 1 


$ 1,143 


$ 144*** 


$ 1,757 


S 54 


2 


1,379 


81 


2,014 


135 


3 


1,449 


129** 


2,133 


164* 


4 


1,520 


109* 


2,199 


94 


5 


1,546 


142** 


2,183 


133 


6 


1,570 


138** 


2,169 


201** 


All quarters 


8,607 


742** 


12,456 


781* 


Sample size ° 




2,287 




2,250 






Other services subgroup 




Quarter 1 


$ 960 


$ 39 


$ 1,677 


S 74 


2 


1.198 


132 


1.951 


104 


3 


1,248 


220** 


2,123 


44 


4 


1.471 


22 


2,199 


44 


5 


1,535 


2 


2,292 


13 


6 


1,548 


42 


2,274 


. 19 


All quarters 


7,960 


457 


12,516 


261 


Sample size " 




1,340 




1J12 



a. Ajsignees ftnd control group mcmben combined. 

* Sutiiticalty tifntfkanl ftt the .10 fevel. ** m the .05 tevel, *** at the .01 level (tvh^o-uUed lect). 
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In contrast to the sustained, positive impact on earnings in the OJT/JSA subgroup, the 
program appears to have had only a short-lived effect on the earnings of adult women, and 
virtually no effect on the earnings of adult men, in the other services subgroup (bottom 
panel). JTPA had a significant impact on women's earnings of $220 in the third quarter, 
followed by much smaller, insignificant gains in the later quarters. The estimated impacts 
on hours worked quarterly (not shown) mirrored this pattern— possibly reflecting quicker 
placement in jobs that were similar to those the female assignees would have eventually 
found without access to JTPA. For men in the other services subgroup, neither the 
estimated impacts on quarterly earnings nor the estimated impacts on hours of work (not 
shown) were statistically significant. 

Overall, then, JTPA led to modest, statistically significant earnings gains in at least one 
quarter for adult women in all three service strategies. The timing of impacts was very 
different across the subgroups, however, and significant for the follow-up period as a whole 
only in the OJT/J'.A subgroup. Significant impacts on the earnings of adult men were 
concentrated exclusively in the OJT/JSA subgroup. 

I: is important to iterate that the adults in the three service strategy subgroups differed 
not only in the services they received, but also in their personal characteristics. Program 
intake staff tended to recommend the most employable applicants for the OJT/JSA service 
strategy. This difference is evident not only in the data on baseline characteristics of the 
three subgroups (not shown here) but also in the earnings of control group members over 
the follow-up period, shown in columns (1) and (3) of Exhibit S.6. These figures indicate 
that in the absence of program services women recommended for OJT/JSA would have 
earned substantially more than those recommended for classroom training and somewhat 
more than those recommended for other services. Among men the more job-ready 
applicants tended to be recommended for either OJT/JSA or other services; those male 
control group members recommended for classroom training earned somewhat less over 
the follow-up period than either of the other two subgroups. 

Because of these differences in the three subgroups, one cannot extrapolate the impacts 
for one service strategy subgroup to the women or men served by another. We cannot, 
for example, conclude that the program outcomes for adult men in the classroom training 
subgroup would have been better if instead they had been recommended for the OJT/JSA 
service strategy. We can only determine which service strategies were effective/or those 
applicants recommended for them. Whether another service strategy would have been 
more effective cannot be determined on the basis of this study, since we did not observe 
alternative service approaches applied to comparable participant populations. 

It is also important to bear in mind that the costs, as well as the impacts, of the three 
service strategies were likely to have varied, as may the longer term impacts. In our final 
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report on this study we will present an analysis that compares the costs of Title II-A to its 
impacts over a longer follow-up period. 

Impacts on Earnings, by ETHNicrrv and Barriers to Employment 

In addition to the three subgroups defined based on service strategy recommendations, v ^ 
estimated program impacts on the 18-month earnings of a number of other subgroups of 
women and men, defined in terms of personal characteristics measured upon their appli- 
cation to the program. These estimates helped us determine whether the impacts of the 
program were concentrated within certain groups of interest to policymakers and program 
planners or broadly distributed across all adult women or men. In this Executive Summary 
we present the results for two such key subgroups: the major ethnic groups, and groups 
facing different barriers to employment 

Exhibit S.7 presents the estimated program impacts on the earnings of white, black, 
and Hispanic women (column 3) and men (column 6). Among women the estimated 
impacts appear to have differed noticeably by ethnic group, with white women showing 
significant earnings gains of $723 over the 18-month follow-up period; black women, an 
insignificant earnings gain of $457; and Hispanic women, an insignificant loss of $-414. 
Moreover, separate tests of the statistical significance of the differences among these 
impact estimates (not shown) indicate the differences were statistically significant at near- 
conventional levels. The estimated impacts for adult men also differed by ethnic group, 
but neither the estimated impacts for individual ethnic groups nor thedifferences in impacts 
among the subgroups were statistically significant and therefore could have arisen by 
chance. 

In an attempt to narrow the range of possible explanations for the differences in 
estimated impacts for women in different ethnic groups, we estimated adjusted impacts that 
controlled for differences in the distributions of these subgroups across study sites and 
across service strategy subgroups . When we controlled for differences in the distributions 
of the three ethnic groups of women across the study sites, the estimated impacts for these 
groups were not significantly different from one another. This finding suggests that the 
differences in estimated impacts among women in different ethnic groups are in part 
attributable to differences in the distributions of these groups across sites. In addition, 
given the extreme concentration of Hispanic women in a few sites we cannot reliably 
distinguish negative effects on Hispanic women as an ethnic group from negative effects 
on all women in one or more of the sites in which Hispanic women were concentrated. 



10. Other key subgroups examined in the report include those defined by work and uaining histories, 
public assistance histories, household income and composition, public housing status, and age. 
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Exhibit S. 7 Impacts on the IS-Month Earnings of Major Ethnic Groups: Aduh JTFA 
Assignees, by Gender 



Ethnic group 




Adult women 






Adult men 




Sample 

a 

Size 

0) 


Control 
mean 

<2) 


Impact per 
assignee 
(S) 


Sample 
size° 
(4) 


Control 
mean 
(5) 


impact per 
assignee 
(6) 


White, non-Hispanic 


3,541 


$ 8,007 


S 723*** 


2,668 


% 12,929 


% 625 


Black, non-Hispanic 


1.981 


6,829 


♦57 


1,155 


10,931 


957 


Hispanic 


744 


6,775 


^14 


400 


13,555 


-741 


Full sample * 


6,474 


7,488 


539*** 


4,419 


12,306 


550 



a. Assigaees and coDiroI {roup members combined. 

b. Including ihc three nwyor ethnic jroupj and American Indians, Alaskan Natives, Asians, and Pacific Islanders. 
• Sutisiically significant at the . 10 level, ** at the .05 level, at the .01 level (two-tailed test). 



To determine whether the effects of the program varied with the degree of labor market 
disadvantage, we also estimated impacts on earnings for subgroups defined by three 
barriers to employment: welfare receipt, limited education, and limited recent work 
experience}^ The first three pairs of rows of Exhibit S.8 show the estimated impacts on 
earnings for women and men facing each of these barriers and for those who were not. 

The mean i8-month earnings of control group members, shown in columns 2 and 5 of 
the exhibit, illustrate that these barriers were indeed serious obstacles to employment. 
Control group members in all three subgroups facing these barriers earned much less over 
the follow-up period than those who were not. 

Among both women and men the estimated impacts tended to be larger for those not 
facing the barriers in question, although among women the differences in impacts between 
those facing and those not facing a particular barrier were smaller than the differences 
among men. Separate tests for the significance of these differences between each pair of 
estimates indicated, however, that any differences shown here may have arisen by chance. 

Because some persons who were facing one of these barriers to employment may also 
have been facing one or both of the other barriers, these subgroups overlap to some 
degree. To achieve a clearer distinction among the subgroups in terms of the overall 
difficulty of becoming employed, the bottom panel of Exhibit S.8 categorizes the women 



1 1 . Welfare receipt is defined as receiving Aid to Families with Dependent Children (AFDC), General 
Assistance, or any other cash welfare benefits upon application to JTPA. Limited education is defined as lack 
of a high school diploma or GED certificate; limited recent work experience is defined as having worked Ics^ 
than 13 weeks in the year prior to application to JTPA. These three measures of baniers to employment arc 
similar to those used in other recent studies of JTPA programs (sec U.S. General Accounting Oflice, 1989). 
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Exhibil S. 8 Impacts on the JS-Month Earnings of Subgroups Facing Selected Barriers to 
Employment: Adult JTPA Assignees, by Gender 







Adult women 






Adult men 




Barrier to 
einploymeni 
(in italic) 


Sample 
size * 


Control 
mean 

(2) 


Impact per 
assignee 


Sample 
size'' 
(4) 


Control 
mean 

(5) 


impact pfr 
assignee 
(6) 


Receiving cash welfare 


3,500 


$ 5,492 
8,965 


$ 387 
697*** 


611 
3,788 


$ 9,541 
13,032 


S -A6 
624* 


No high school diploma 
or GEO cenificaie 


1,731 


6,072 


416 


1,249 


10.353 


398 


High school diploma 
or GED ccrtificAtc 


4.316 


8,064 


681*** 


2.873 


13,335 


878** 


Worked less than 13 
weeks in past 12 mos. 


3,022 


5,555 


511** 


1,614 


10.478 


-210 


Worked 13 weeks or 
more in past 12 mos. 


2,622 


9.956 


668** 


2.392 


14.320 


787* 


Number of barriers 
None of the above 


1,361 


10,971 


909** 


1.465 


15.142 


1,203** 


One of the above 
Two of the above 


1,655 
1,435 


7,950 
5,756 


802** 

379 


1,550 
617 


12.184 
9.044 


194 
30 


All three of the above 


488 


3,703 


-213 


116 


8,595 


-146 


Full sample 


6,474 


7,488 


539*** 


4,419 


12.306 


550 



a Assignees and cooirol {roup members combioe^i. 

• Sutisiictlly iignifioml at ihc .10 level. at the .05 level, *•* at the .01 level (two-tailed test). 



and men in the sample by the number of these barriers they were facing. Again, the 
average earnings of the control groups indicate that this categorization is strongly 
predictive of what JTPA assignees would have earned without the program: control group 
earnings fall steadily as the number of barriers rises. 

For both women and men the impacts were the largest in the subgroup facing none of 
the three barriers. For neither women nor men, however, were the differences in impacts 
among subgroups statistically significant; thus, these differences may merely reflect 
sampling error. 

Summary and Comparison wfth Previous Findings 

Overall , JTPA Title II-A had a mod est positive impact on the earnings of adult women over 
the follow-up period: on average, a significant gain of $539 over the 1 8 months following 
their application. The estimated earnings gain for men was similar ($550) but was not 
statistically significant. These overall averages mask substantial variation in both the 



12. Among the aduli female subgroups, for example, there is a 46 percent chance of finding differences ai 
least as large as those shown here even if there were no true differences in impacts among the subgroups. 
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magnitude and time patterns of program impacts among subgroups of women and men. 
however. 

When adult wome.i are categorized by the service strategy recommended by program 
intake staff, the only ones to experience a statistically significant earnings impact over the 
follow-up period as a whole were those in the OJT/JSA subgroup, with a gain of $742, 
Women in this subgroup enjoyed consistently positive, statistically significant earnings 
increases of $109 to $144 in five of the six follow-up quarters. Women in the classroom 
training subgroup experienced an earnings loss in the first calendar quarter of the follow- 
up period, followed by growing positive impacts, and culminating in significant impacts 
of $144 and $188 in the fifth and sixth quarters. Program impacts on the earnings of 
women in the other services subgroup were significant only in the third quarter, when 
these women gained $220, on average; impacts for this subgroup were negligible in 
subsequent quarters. 

Impacts for adult men were similar in magnitude to those for women, although they 
were less frequently statistically significant. As with the women, only those in the 
OJT/JSA subgroup enjoyed significant earnings gains (of $78 1) over the follow-up period 
as a whole. Estimated impacts on the earnings of men in the classroom training and other 
services subgroups were never statistically significant, either for the foliow-up period as 
a whole or for individual quarters. 

These impact estimates are similar in magnitude to those found in the few previous 
evaluations that have used rigorous experimental designs. For example, studies of state 
work-welfare programs for women in the early 1980s found significant positive impacts 
in the first two years after random assignment that ranged up to about $250 per quaner.^^ 
Evaluations of demonstration programs for displaced workers in Texas and New Jersey 
found simil ar impacts on the earnings of men— that is, in the same range but not statistically 
significant— in the first year after random assignment.'* 

Comparisons with the results of earlier studies are complicated, however, by the fact 
that tl. programs involved in those studies provided somewhat different services from 
those in JTPA and served primarily subpopulations such as welfare recipients and 
(for men) displaced workers and ex-addicts. Moreover, theprograms for women examined 
in earlier studies were, unlike JTPA, mostly mandatory, and yet had lower rates 
of participation in employment and training services than those of our study sample. 



13. See Gucron and Pauly (1991) 

14. See Bloom (1990) and Corson ei al. (1989). 
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Finally, when adult women in the National JTPA Study sample were classified by 
ethnic group, differences in estimated impacts on earnings were emerged, with white 
women experiencing greater gains than minority women, particularly Hispanic women. 
Fuaher tests revealed, however, that these differences in impacts may well have been due 
to the concentration of Hispanic women in a few sites that experienced impacts 
substantially below the average for all women. There were no significant differences in 
impacts on the earnings of adult men by ethnic group. Impact estimates for adults who 
were and were not subject to various barriers to employment were not statistically 
significantly different from each other. But the pattern of estimates for these subgroups 
suggested that JTPA produced larger positive impacts for adults with fewer labor market 
barriers. 

Findings for Subgroups of Female and Male Out-of-School Youths 

Out-of-school youths in the study sample were classified into the same three service 
strategy subgroups as those used to classify adults: classroom training, OJT/JSA, and 
other services. These subgroups were based on the JTPA services recommended for 
sample members by program intake staff before random assignment. 



Service Strategies Recommended 

The service strategies recommended for youths reflect a difference in emphasis between 
JTPA Title II- A programs for youths and those for adults. Programs for adults emphasize 
employment, as evidenced by the fact that program performance standards for adults are 
based largely on job placement rates. In contrast, programs for youths emphasize abroader 
range of outcomes, with performance standards for youths based in part on **positive 
terminations," which include not only job placements but also participation in further 
training and attainment of specific job competencies. 

A comparison of Exhibit S.9 and the earlier Exhibit S.3 indicates that youths were far 
less likely than adults to be recommended for the OJT/JSA strategy, especially if we 
compare female youths with female adults and male youths with male adults. Of the three 
service strategies OJT/JSA places the greatest emphasis on immediate employment; thus, 
this difference between youths and adults reflects the difference between JTPA programs 
for the two age groups. In addition, youths were much more apt than adults to be 
recommended for the other services strategy, which, as discussed below, also differed 
between the two age groups in the mix of program services received. 

Service strategy recommendations also differed between female and male youths 
themselves. Female youths were more likely than male youths to be recommended for 
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Exhibit S,9 Service Strategies Recommended: 

Out-of-School Youth JTPA Assignees, 
by Gender 





Female 


Male 




youths 


youths 


Service strategy 


(1) 


(2) 


Classroom training 


44.3% 


29.9% 


OJT/JSA 


232 


32.9 


Other services 


32.5 


37.3 


Sample size 


1,814 


1,436 



classroom training (44 percent versus 30 percent, respectively) and less likely than male 
youths to be recommended for OJT/JSA (23 percent versus 33 percent). The genders were 
about equally likely to be recommended for other services (33 percent versus 37 percent). 



Enrollment Rates and Duration, by Service Strategy Subgroup 

Enrollment rates overall were comparable to those for adults, with 65 percent of the female 
youth treatment group and 67 percent of the male youth treatment group becoming enrolled 
in JTPA Title II-A at some time during the 18-month follow-up period. Treatment group 
enrollment rates were highest in the classroom training subgroup (71 percent for females 
and 75 percent for males). The lowest enrollment rates were in the OJT/JSA subgroup 
(57 percent for females and 58 percent for males). The other services subgroup fell 
between these two extremes, with enrollment rates of 63 percent for female and 68 percent 
for male youth treatment group members. 

Out-of-school youths who enrolled in JTPA stayed in the program slightly longer than 
their adult counterparts, with the median duration of enrollment at 3.9 months for female 
youths (versus 3.6 months for adult women) and at 3.1 months for male youths (versus 
2.5 months for adult men). Thus, the median duration of enrollment was also slightly 
longer for female than for male youths. The service strategy subgroup with the shortest 
enrollments was OJT/JS A, with a median of about 2 months for both females and males; 
the classroom training subgroup had the longest enrollments, at 5.5 months for females 
and 4.6 months for males. The median for the other services subgroup was about 3 months 
for both target groups. 
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Exhibit S, JO Key JTPA Services Received by Treatment Group Members 
Who Were Enrolled in the Program: Out-of-School Youths, 
by Gender and Service Strategy Subgroup 





% of enroUees receiving 
one or both services 




Female Male 


Key services 

in service strategy subgroup 


youths youths 

(1) a) 




Classroom training subgroup 


Classroom training in occupational 




skills/basic education ^ 


86.1% 80.47c 




OJT/JSA subgroup 


On-the-job training/ 




job search assistance 


84.8% 84.57c 




Other services subgroup 


Basic education/ 




miscellaneous 


79.5% 83.2% 


Sample size 


2,288 959 



Q. 'Basic educatioa' iflcludes Adull Basic Educatioo <AB£). faish school or Generml Educalional 



Developmcat (GED) preptrtlioa, and English as a Secood Laof uage <ESL). 
b. 'Mitceilaneous' iociude* asseaamcal, job*re«diaeaa trainiag, customized tnining, vocatiooal 
expioratioD, job thadowia|, and tiyout etnploymem, among other services. 



Services Received, by Service Strategy Subgroup 

Exhibit S.IO shows the percentage of enrollees in each service strategy subgroup who 
received one or both of the key services in that service strategy. About 86 percent of female 
youth enrollees and about 80 percent of male youth enrollees recommended for classroom 
training received classroom training in occupational skills, basic education, or both. 
About 85 percent of the female and male youth enrollees in the OJT/JSA subgroup received 
on-the-job training, job search assistance, or both. And about 80 percent of the female 
and 83 percent of the male youth enrollees in the other services subgroup received basic 
education, miscellaneous services, or both. 

The only obvious difference between the mix of JTPA services received by youths and 
the mix received by adults was in the other services subgroup. Whereas adult enrollees 
in this subgroup received mainly job search assistance and miscellaneous services (Exhibit 
S.4), the youth enrollees received mainly basic education and miscellaneous services- 
further evidence, as noted earlier, that JTPA emphasizes immediate employment for adults 
more than it does for youths. 
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As with the adults in our sample, the estimated program impacts on youths reponed 
below reflect differences in the employment and training services received by treatment 
group members, who had access to JTPA, and the services they would have received if 
they had been excluded from the program, as measured by data on control group members. 
And as with adults, the size of these treatment-control group differences in service receipt 
varied by service strategy subgroup. 

In the classroom training subgroup about 48 percent of the female youths and 43 
percent of the male youths in the treatment group received classroom training in 
occupational skills, whereas only 31 percent of the female youths and 22 percent of the 
male youths in the control group received this service. In the OJT/JSA subgroup about 
30 percent of the females and 31 percent of the males in the treatment group received on- 
the-job training, while less than 1 percent of both females and males in the control group 
received the service. 

We were unable to measure the control group's receipt of miscellaneous services—the 
most common category of services received by youth treatment group members in the other 
services subgroup, at 29 percent for females and 35 percent for males. It is therefore not 
possible to determine the treatment-control group difference in service receipt for this key 
service in the subgroup. The service differential was small, however, for basic education, 
the other key service received by youth treatment group members who were recommended 
for the other services strategy. About 23 percent of female youths and 14 percent of male 
youths in the treatment group received basic education, while 19 percent of the females 
and 12 percent of the males in the control group received it. 

Thus, JTPA produced a noticeable increment in service receipt in the two service 
strategy subgroups for which we could measure the differences: classroom training and 
OJT/JSA. In the third, other services, we could not measure the treatment-control group 
differential for the most common service received by the treatment group and found only 
a slight differential in the case of the other key service. In terms of the average number 
of hours of services received, JTPA produced a modest increase for the two subgroups for 
which we could measure this effect. Female and male youths in the classroom training 
subgroup received, respectively, 187 and 127 more hours of classroom training in 
occupational skills than they would have if JTPA were not available, while female and 
male youths in the OJT/JSA subgroup received, respectively, an additional 105 and 128 
hours of on-the-job training. 

Impacts on Educational Attainment, by Service Strategy Subgroup 

Differences in the services received by youths in the treatment and control groups produced 
differences in the rate at which high school dropouts in these groups attained a high school 
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Exhibit S.J I Impaas on Anainment of a Training-Related High School Diploma or GED 
Certificate: Out-of-Sdiool Youth JTFA Assignees Who Were High School 
Dropouts, by Gender 







Female youths 






Male youths 




Service 


% attaining HS/GED 


Impaa, in 


% attaining 


HS/GED 


Impact, in 


strategy 


Assignees 


Controls 


% poirus 


Assignees 


Controls 


% points 


subgroup 


(I) 


(2) 


(3) 


(4) 


(5) 


(6) 


Classroom 














training 


32.9% 


16.6% 


16. 4*** 


27.3% 


18.3% 


9.0* 


OJT/JSA 


9.8 


6.0 


3.8 


14.9 


4.9 


10.1*** 


Other services 


31.7 


21.0 


10.7** 


26.1 


16.9 


9.1*- 


All subgroups 


28.6 


16.6 


11.9*** 


23.9 


14.0 


9.9**- 


Sample size " 






1.050 






955 



a. Aitignees and control croup members who were hi{h school dropouts. 

* Sutislicaliy sienificanl at the .10 icvel. ** at the .05 level, *** at the .01 level (two<uUed test). 



diploma or GED certificate. S .ice half of the female youths in the study sample and three- 
filths of the male youths were high school dropouts, impacts on their educational 
attainment represent an important result of the program. 

As shown in the fourth row of Exhibit S.ll, among control group members who were 
dropouts 17 percent of the female youths and 14 percent of the male youths both enrolled 
in a school or training service and received a high school diploma or GED certificate at 
some time during the 18-month follow-up period. Among the corresponding treatment 
group members, however, 29 percent of the female youths and 24 percent of the male 
youths subsequently attained a training-related high school credential. The program 
impact in both cases was highly significant. Impacts were also statistically significant for 
male youths in all three service strategy subgroups and for females in the classroom 
training and other services subgroups— the two service strategy subgroups that focused the 
most on basic education. The impact was particularly striking for female youths in the 
classroom training subgroup. 



Impacts on Earnings, by Service Strategy Subgroup 

As noted at the outset, the estimated program impact on the earnings of female youths 
overall was negligible; the impact on male youths overall was substantially negative, but 
that impact was largely concentrated among those male youths who reported having been 
arrested between their sixteenth birthday and random assignment. Exhibit S. 12 provides 
a moredetailed understanding of these findings by presenting estimates for the three service 
strategy subgroups of youths during each of the six quarters of the follow-up period. 
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Exhibit S.J 2 Impacts on Quarterly and IS-Monih Earnings: Out-of-School Youth 
JTPA Assignees, by Gender and Service Strategy Subgroup 



Female youths Male youths 





Control 


Impact per 


Control 


Impact per 




mean 


assignee 


mean 


assignee 


Period 


(J) 


(2) 


(3) 








Classroom training subgroup 




Quarter 1 


S 742 


% -210*** 


S 1,226 


S -300** 


2 


909 


- 189*** 


1,345 


96 


3 


1,052 


-150* 


1,655 


- 2 


4 


991 


24 


1,773 


0 


5 


1,047 


70 


1,889 


-56 


o 


1 , lyo 


-87 


1,895 


A 


All quarters 


5,936 


•542 


9,783 


- 259 


Sample size " 




1,045 




526 






OJT/JSA subgroup 




Quarter 1 


S 1.002 


$ 149 


$ 1,651 


S -57 


2 


1,074 


203* 


1,988 


- 219 


3 


1,252 


97 


2,197 


- 302* 


4 


1,363 


3 


2,160 


- 203 


5 


1,368 


103 


2,316 


- 192 


6 


1.562 


-146 


2,452 


- 339** 


All quarters 


7.620 


410 


12,765 


- 1,313* 


Sample size ^ 




545 




615 






Other services subgroup 




Quarter 1 


$ 653 


$ 43 


$ 1,362 


S -285** 


2 


909 


-68 


1.457 


- 121 


3 


1.023 


-96 


1,605 


-218 


4 


1,047 


-52 


1.751 


-276* 


5 


U093 


-41 


1,766 


- 114 


6 


1,001 


55 


1,899 


- 292** 


All quarters 


5,726 


-158 


9,839 


- 1,305* 


Sample size ^ 




710 




607 



a AMt|neef And conifol |roup membcn combined. 

* StA(udc«Uy tigniTicftnt at the .10 tevcl. ** at the .OS level. *** it the .01 level (^vo•uUed tat) 
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In the female classroom training subgroup impacts on earnings were negative and 
statistically significant during the first three follow-up quarters. As with adult women, 
these initial losses probably reflect the earnings forgone by treatment group members 
while they were attending classes. Unlike the experience of adult women, however, female 
youths in classroom training did not experience any significant increases in earnings later 
in the follow-up period. Hence, the earnings female youths lost uhile participating in 
classroom training were not offset by a payback period, at least not by the end of the 18- 
month follow-up. 

Female youths in the OJT/JSA subgroup experienced a different pattern. The initial 
impacts on their earnings were moderately positive (and statistically significant in the 
second follow-up quarter), which may reflect an initial boost in employment produced by 
on-the-job training, job search assistance, or both. But these short-run gains were not 
sustained over time. 

Program impacts on the earnings of female youths in the other services subgroup were 
negligible in all six follow-up quarters. In other words, the mix of predominantly 
miscellaneous services and basic education that JTPA provided to this subgroup had linle 
or no impact. 

The impact estimates for male youths in the classroom training subgroup were similar 
to those for their female youth counterparts. Impacts were substantially negative and 
statistically significant in the first follow-up quarter, again, perhaps reflecting the costs 
of being in class instead of employed. And as with female youths, the later follow-up 
quarters brought no earning increases large enough to offset the initial loss. 

Impacts on male youths in the OJT/JSA subgroup were negative in all six follow-up 
quarters. Over the follow-up period as a whole the OJT/JSA strategy yielded a statistically 
significant earnings loss of $-1,3 13, or -10.3 percent of the corresponding control group's 
mean earnings. This loss reflected mainly an estimated -8.5 percent program-induced 
reduction in the average number of hours worked by male youths; average hourly earnings 
among those who worked were largely unaffected by the program (not shown in the exhibit). 

Male youths in the other services subgroup experienced an estimated earnings loss of 
S-1,305, or -13.3 percent of what their earnings would have been without access to JTPA. 
This loss reflected mainly a -9.7 percent reduction in the average number of hours worked, 
although average hourly earnings when working were also reduced by an estimated -4.0 
percent (not shown). 



1 5 . The percentage impacts on hours worked and on earnings per hour worked do not sum exactly to the 
percentage impact on total c imings because the relationship between total earnings and its components is 
multiplicative, not additive. 
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For these last two service strategy subgroups of male youths, it therefore appears that 
the negative program impact on earnings reflected mainly a negative program impact on 
the number of hours worked, as opposed to a negative impact on the hourly earnings of 
those who worked. The next subsection will also demonstrate that the negative impacts 
on earnings for the OJT/JSA and the other services subgroups of male youths are 
attributable primarily to a large negative estimated impact on the earnings of those male 
youths with a previous arrest who were recommended for each of these two service 
strategies.^* 

It is important to bear in mind that although this analysis by service strategy subgroup 
is illuminating, one cannot interpret the findings for one service strategy subgroup as 
having direct implications for the youths recommended for one of the other two service 
strategies. Again, we can only determine which service strategies were effective for those 
applicants recommended for them, because the three service strategy subgoups differed in 
the personal characteristics of their members. 

IMPACTS ON Earnings, by ETHNicrrv, Barriers to Employment, and 
Reported Arrests 

The estimated program impacts on earnings for out-of-school youths did not vary 
systematically with the ethnic backgrounds of sample members or with the barriers to 
employment they faced when they applied to JTPA. 

Exhibit S.13 presents the estimated program impacts on white, black, and Hispanic 
youths. The impact estimates for female youths did not differ substantially by ethnic 
group, and no ethnic group experienced a statistically significant impact. In addition, 
separate tests of the statistical significance of the differences among the impacts on these 
groups (not shown) confirm the lack of a differential effect of JTPA. For the male youths 
there were differences in estimated impacts among the three ethnic groups, but these 
differences were not statistically significant and may therefore have been due to chance 
(test not shown). 

Exhibit S.14 presents the estimates for subgroups of youths defined in terms of the 
three specific barriers to employment investigated for adults: welfare receipt, limited 
education, and limited recent work experience. As was the case for adults, these barriers 
represented serious obstacles to employment for youths, as evidenced by the fact that 



16. Note that the much smaller and statistically insignificant estimated impact on the earnings of male 
youths in the classroom -training subgroup is not attributable to this subgroup's havmg a substantially smaller 
proportion of previous arrestees than ihc other two service strategy subgroups (which ii did not). 
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Exhibit S. 23 Impacts on the IS-Monih Earnings of Major Bhnic Groups: Out-of-School 
Youth JTPA Assignees, by Gender 







Female youths 






Male \oui}is 






Sample 
size * 


Control 


Impact per 


Sample 


Control 


Impact per 




mean 


assignee 


size 


mean 


assignee 


Bhnic group 


(1) 


a) 


(S) 




(5) 


(6) 


While, non-Hispanic 


1.148 


S 7.076 


S -122 


946 


S 12.550 


S -1,333** 


Black, non-Hispanic 


749 


5.601 


-135 


522 


8.164 


75 


Hispanic 


366 


5.019 


-554 


24S 


10.126 


-1,238 


Full sample * 


2,300 


6.225 


-182 


1,748 


10,736 


- 854** 



a. Afsigoeet and control group monbert combined. 

b. Including the throe major ethnic groupt and American Indians. Alatkan Natives, Asians, and Pacific Islanders. 
• Sutiilically iignificanl at ihc .10 level, •* at the .05 level, at the .01 level (two-uiled icst). 



control group earnings drop markedly as the number of barriers increases (bottom panel 
of the exhibit). 

There was no statistically significant relationship, however, between the number or 
nature of these employment barriers and the effect of JTPA on out-of-school youths. 
Among female youths, in particular, there v/as little difference between the estimated 
program impact on sample members who faced each of the three employment barriers and 
those who did not face that barrier. Furthermore, there was no clear pattern in the 
relationship between the estimated program impacts and the number of employment 
barriers faced. Tests for significant differences in impacts among subgroups (not shown) 
revealed none that was statistically significant. 

Among male youths the differences between the impact on sample members who faced 
a particular employment barrier and those who did not appear to have been more 
substantial. For male youths with limited education or limited recent work experience, 
JTPA appears to have reduced the earnings of those facing one of these two barriers by 
more than it reduced the earnings of those who did not (top panel, column 6). In addition, 
the more barriers faced, the more JTPA seems to have reduced earnings over the follow- 
up period. None of these differences in impact estimates between or among the subgroups 
were statistically significant, however, and so the patterns they imply are only suggestive 
and may in fact be due to chance. Moreover, the difference in impacts was in the opposite 
direction for male youths receiving welfare and those not receiving welfare. 

Again, the most striking subgroup difference for youths was between the impacts 
estimated for male youths who had been arrested before and those who had not. As shown 
in Exhibit S.15, on average, male youths with a previous arrest experienced a highly 
significant $-3,038 program-induced earnings loss during their 18-month follow-up 
period. In contrast, male youths without a previous arrest experienced an insignificant 
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Exhibit S.14 Impacts on the IS-Month Earnings of Subgroups Facing Selected Barriei 
Employmeni: Oui-of-School Youth JTPA Assignees, by Gender 





Female youths 






Male \ouths 




Barrier to 
employment 

fin italic) 


Sample 

size' 

(1) 


Conirol 
mean 

(2) 


Impact per 
assignee 

(3) 


Sample 
size" 

(4) 


Control 
tnean 

rS) 


Iinpcjct per 
assignee 

(6, 


Receiving cash welfare 
No cash welfare 


701 
1,412 


5 4,397 
7,174 


$ -391 
-1S4 


185 
1,374 


S 8,815 
11,292 


S -56 
-1,020" 


So high school diploma 
or GED certificate 


1,047 


4,192 


23 


947 


10.087 


.1,144» 


High school diploma 
or GED certificate 


1,146 


8,055 


^37 


730 


11,612 


-420 


Worked less than 13 
weeks in past 12 mos. 


1,235 


4,425 


-31 


754 


8,616 


^1,286" 


Worked 13 weeks or 
more in past 12 mos. 


829 


8,886 


-255 


842 


12.808 


-832 


Number of barriers 
None of the above 
One of the above 
Two of the above 
All three of the above 


545 
790 
675 
281 


9,964 
6,552 
4,486 
2,189 


-260 
-236 
-451 
659 


475 
733 
455 

81 


13.352 
10,810 
8,520 
7.642 


-459 
-695 
-1,242 
-1,278 


Full sample 


2,300 


6,225 


-182 


1,748 


10.736 


-854 « 



a. Assignees and conirol croup membcn combined. 

♦ Sutisiicilly significant at the .10 level. aithc .05 level, at the .01 Icvd (two-uilcd icsl). 



S-224 earnings loss. The difference between these two impact estimates was highly 
significant and did not change when we controlled for the distributions of the two 
subgroups across the study sites and service strategy subgroup n. 

Moreover, this difference appeared in all six follow-up quarters, all three service 
strategy subgroups, and 13 of the 15 study sites where youths were included in the sample, 
The negative impact on the subgroup of male youths with a previous arrest (25 percent of 
the male youth treatment group) accounts for 82 percent of the program-induced earnings 
loss for male youths overall. 

It is important to note, however, that these large, negative impact estimates, which are 
based on our First Follow-up Survey (the basis for all the impact estimates in this report), 
differ substantially from corresponding impact estimates for male youths with a previous 
arrest that are based on earnings data from an alternative data source, namely, the 

17. The Oakland sile excluded youths from the study, yielding a toial of 15 study sites for the youth 
analysis. 
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Exhibit S. 15 Impacts on the JS-Month Earnings of Subgroups With and Withou: a 
Previous Arrest: Out-cf-School Youth JTPA Assignees ^ by Gender 
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• SuiirtictUy lipiificanl «t the .10 level, •• « the .05 level, ••• «i tbc .01 level (two-Uiled lc<). 



administrative records of state unemployment insurance agencies, " Impact estimates based 
on UI earnings data for a subsample of the 18-month study sample suggest there was 
virtually no program impact on the earnings of the previous arrestees among male youths. 
Although there is thus some question about the degree to which JTPA reduced the earnings 
of those male youths with a previous arrest, both data sources agree that the program did 
not increase their earnings, or the earnings of male youths overall. 

We will explore further the differences in the estimates from the two data sources in 
our forthcoming final report. The impact estimates from the two data sources do not, 
however, differ appreciably for adult women, adult men, female youths, or those male 
youths who did not report a previous arrest. 

Summary and Comparison wrrH Previous Findings 

The preceding analysis has shown that JTPA Title II-A did not appreciably affect the 
earnings of female out-of-school youths. On average, the program reduced total earnings 
during the 1 8-month follow-up period by $-182 per female youth assignee (treatment group 
member), but this estimated effect was not statistically significant. Nor were the impact 
estimates statistically significant for female youths in each of the three service strategy 
subgroups or in any of the subgroups defined by personal characteristics. 

The findings for male out-of-school youths are very different. On average, JTPA 
reduced the estimated earnings of this target group by a statistically significant $-854 over 
the 18-month follow-up period. But most of this negative estimated impact was 



18. Appendix E examines this issue As discussed there, the impact fmdings for male youths with a 
previous arrest difler between the two data sources because earnings data on the treatment group and the 
control group of male youtji arrestees dificr between the two data sources 
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concentrated among the 25 percent of male youths who had a previous arrest. Hence, tor 
most male youths (the 75 percent who reported no previous arrest) the program appeared 
to have a negligible effect, as was the case for female youths. 

The findings for out-of-school youths in this study are not inconsistent with those from 
the two existing experimental studies of employment and training programs for out-ot- 
school youths.*^ The first, the youth component of the National Supported Work 
Demonstration, evaluated an intensive work experience program (Manpower Demonstra- 
tion Research Corporation, 1980); and the second, JOBSTART, evaluated intensive 
education, employment, and trainingservicesprovided through JTPA (Cave and Doolittle, 
1991). The Supported Work study found negligible post-program impacts on the earnings 
of youth participants, most of whom were male. JOBSTART found negligible shon-term 
impacts for female youths and large negative short-term impacts for male youths, 
mirroring the findings of the present study. 

Both JOBSTART and the youth component of Supported Work targeted seriously 
disadvantaged youths, who make up only a portion of the out-of-school youth population 
targeted by JTPA Title II-A programs. And Supported Work provided far i.iore intensive 
services than are typically available from JTPA. Thus, the three studies of employment 
and training programs for youths focus on different target groups and program services. 

Nevertheless, none of these studies indicates that the programs examined were able 
to improve the earnings prospects of disadvantaged youths; and two of *he three studies 
found that the programs actually reduced the earnings of male youths, at least in the short 
term. The experimental findings to date are therefore cause for concern. 



Implications of the Findings 

The National JTPA Study is based on an examination of 16 study sites, which are not a 
probability sample of all JTPA service delivery areas and which, despite their diversity, 
may not be representative of the nation. Nevertheless, to the extent that the findings in 
this report apply to other localities, they have important policy implications. 

The study has shown that JTPA Title II-A is helping to raise the earnings of many of 
its participants, especially adults, but it has also identified several groups for whom the 
program is having no effect or even adverse effects. In particular, the Title II-A programs 

1 9. Although many other cmploNincnt and training programs for youths have been studied in the past, the 
findings obtained provide little reliable information because of the methodological problems endemic to the 
nonexperimental research designs that were used. See the review in Betsey. Hollister, and Papageorgiou 
(1985). 
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Studied failed to raise the average earnings of out-of-school youths in general, and ihey 
reduced the average earnings of male out-of-school youths who reported having been 
arrested between their sixteenth birthday and random assignment. 

But although this analysis has identified groups not being adequately served by the 
program, we cannot use these findings to prescribe ways to serve them better. The study 
was designed to observe only the impacts of JTPA as it was operated during the study 
period, not alternative ways of serving ihe same population. 

Finding ways to improve program performance for those groups negligibly or 
adversely affected by the current program will require experimentation with a range of 
alternative service strategies for those groups and rigorous evaluation of their impacts. 
We cannot overemphasize the importance of rigorous evaluation of new approaches to 
serving these groups. Experience has demonstrated that simply trying out alternative 
program strategies without rigorous evaluation is not enough. As a National Research 
Council report concluded in reviewing some 400 reports on a wide range of youth 
employment and training demonstrations, **Despite the magnitude of the resources 
ostensibly devoted to the objectives of research and demonstration, there is little reliable 
information on the effectiveness of the programs in solving youth employment problems*' 
(Betsey, Hollister, and Papageorgiou, 1985). To address this deficiency, the authors 
recommended greater reliance on field experiments with random assignment. 

Indeed, the reason it is difficult to draw conclusions from studies that do not use 
random assignment is clear from our findings on the control groups in this study. The 
patterns of control group earnings over the 18-month follow-up period demonstrate that 
even without access to JTPA both adults and youths would have experienced a growth in 
earnings, and their earnings would have varied substantially across the three service 
strategies. In other words, if one looks only at the post-program earnings and employment 
of program participants, one can easily mistake patterns of outcomes that would have 
occurred anyway for impacts of the program. 

Finally, although the findings presented here clearly reveal a need for some program 
changes, the full findings of the National JTPA Study have not been obtained. Our 
forthcoming final report will extend the analysis in several ways. First, we will estimate 
program impacts on earnings, employment, and educational attainment over a longer 
follow-up period. Growth or decline in the impacts during the period beyond 18 months 
could materially alter the differences in estimated impacts among target groups, service 
strategy subgroups, and other key subgroups that we have observed thus far. Second, we 
will also include estimated impacts on the receipt of AFDC and food stamp benefits. Third, 
and most important, we will compare the impacts and costs of JTPA Title 11-A and its three 
service strategies, to determine the cost-effectiveness of the program at the 16 study sites. 
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Background: JTPA Title II-A Nationally, 
Previous Research, and the National JTPA Study 



THE National JTPA Study was commissioned by the U.S. Department of Labor 
in 1 986, in response to a Congressional mandate to study the effectiveness of programs 
fimdeil by the Job Training Partnership Act of 1982. ITiis JTPA legislation specified that 
analysis be conducted of the "'increase in employment and earnings for participants, reduced 
support costs, [and] increased tax revenues" (section 454 of the act). 

The National JTPA Study employs a randomized experiment to estimate the impacts of 
JTPA Title II-A programs operated by 16 local service delivery areas (SDAs) in die 
continental United States. Specifically, over a period starting in November 1987 and ending 
in September 1989, the experiment randondy assigned all Title n*A eligible adults and out- 
of-school youths who ^plied to these 16 study sites and were judged sq)propriate for JTPA 
by site staff to one of two groups: a treatment group, whose members were given access to 
program services, and a control group, vdiose members were not allowed to receive program 
services, for a period of 18 months after their random assignment.^ The study compares the 
subsequent earnings, employment, and wel&rt receipt of these two matched groups to obtain 
estimates of Title II-A impacts on die populations served at the sites. 

The decision by the Department of Labor (DDL) to sponsor this type of study was based 
on a growing consensus among researchers at the time that a randomized experiment was 
indeed necessary to achieve valid and reliable evidence of the impacts of employment and 



1 . The period of random assignment was different for each SDA, but the first sample member entered the 
study in November 1987 and the last one entered in September 1989. 
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training programs^ — and on the unanimous reconunendation to the same ejBFect offered by a 
research advisory panel convened by DOL to determine how best to evaluate JTPA programs 
(Stromsdorfer ct al., 1985). 

The 1 6 SDAs that participated in the study represent a broad range of programs, program 
participants, and labor markets. All told, the study*s findings — based on survey data, SDA 
administrative records, and data from state unemployment insurance agencies — ^will provide 
the first valid and reliable evidence of the impacts of JTPA Title II-A programs. The analysis 
focuses on a variety of different groups within the study sample of 20,601 eligible program 
applicants. A first set comprises four main target groups of Title II-A: economically 
disadvantaged adult women and men and female and male out-of-school youths.^ A second 
set comprises groups defined by clusters of specific program services, or service strategies, 
recommended for them by SDA intake staff. TTie study's analysis of these service strategy 
subgroups offers insight into the impacts of different combinations of specific program 
services on the groups of program participants deemed likely to benefit from them. Finally, 
the study also examines impacts on a number of key subgroups defined by individual 
characteristics— such as ethnicity, race, or such barriers to employment as welfare receipt, 
limited education, and limited recent work experience — ^that figure prominently in JTPA 
pohcy debates. 

This report presents estimates of program impacts on the earnings and employment of 
each of these groups over the 1 8 months following random assignment. A companion volume 
(DooUttle, forthcoming) describes how the 16 SDAs operated their Title II-A programs at the 
time of the study and how the randomized experiment was implemented. The other volumes 
in this series are listed at the front of this report. Our forthcoming final report will examine 
iirpacts on earnings, employment, and welfare receipt over a longer follow-up period and 
present a benefit-cost analysis of the local programs studied. 

The remainder of this chapter offers background on the JTPA program nationally, the 
results and limitations of previous research on employment and training program impacts, and 
the more specific goals and objectives of the National JTPA Study. 



The JTPA Title II-A Program Nationally 

The federal government has sponsored job-training programs for unemployed and economi- 
cally disadvantaged Americans for almost three decades. These programs began with the 



2. Sec Fiakcr tnd Maynard (1984); ULoodc (1984); Betsey, HoUistcr, and Papagcorgiou (1985); and 
Buitlcss and Orr (1986). 

3. The study excluded in-school youths, for reasons discussed in Chapter 2. 
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Manpower Development and Training Act of 1962 (MDTA), which was replaced in 1 973 b\' 
the Comprehensive Employment and Training Act (CETA), which, in turn, was replaced in 
1982 by the Job Training Partnership Act (JTPA) — the current federal program. Title II-A 
of JTPA — the focus of the present study — is designated to serve the emplovment and training 
needs of economically disadvantaged adults 22 years of age and older and youths, 16 to 2 1 
years old/ According to its statement of purpose (section 2), Title II-A of JTPA is intended: 

to prepare youth and unskilled adults for entry into the labor force and to 
afford job training to those economically disadvantaged individuals and 
other individuals facing serious barriers to employment, who are in special 
need of such training to obtain productive employment. 

For adults, the program is intended to increase earnings and employment, and reduce 
dependence on welfare. For youths, the program has some^^tot broader objectives, which 
include fostering their attainment of educational credentials and occupational competencies, 
as well as increasing their earnings and employment. 

JTPA was one of the first "New Federalism" programs, which sought to decentralize 
program planning and oversight. As such, it has stimulated wide variation in program content 
and administration. The ability to tailor programs to local needs and opportunities, rather than 
to implement a standard intervention, is fundamental to JTPA. 



Administration 

JTPA Title II-A is funded by the federal government, which spends about $1.8 billion annual 1\ 
to serve roughly a million participants a year (U. S . General Accounting Office, 1991). The 
states coordinate and regulate local JTPA activities, which are administered by county and 
city governments. 

Within this framework the federal government allocates JTPA funds in two parts. The 
largest part, 78 percent of the total, is allocated by a formula directly to the local SDAs 
administering the program.^ The remaining 22 percent is allocated to the states as set-asides 
to promote specific program objectives.* 



4. Some local Tiilc II-A programs also serve M- and 15-ycar-olds. 

5. The fomiula allocates ihe 78 percent of funds in two steps: first to each slate, and then lo the SDAs 
within each state The states, however, have no direct role in this allocation. 

6. These state set-asides are 3 percent for services to older workers. 8 percent to coordinate JTPA programs 
with educational programs, 6 percent for SDA pcrfonnancc incentives, and 5 percent for slate auditing and 
administrative costs. 
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Nationally, there are 649 SDAs, covering every part of the countrj'. Formed by one or 
more local governments, the SDAs operate local JTP A programs with guidance from a Private 
Industry Council. These PICs conq)risc representatives of local businesses, unions, social 
service agencies, and employment and training organizations. 



Services 

SDAs provide specific employment and training services (often tenncd program activities) 
either directly through their own sta£f or by contracting with other local service providers, such 
as public schools, community colleges, proprietary schools, and commimity-based organiza- 
tions. The specific services offered come in many different fonns, but they generally fisdl under 
one of six basic categories: 

• classroom training in occupational skills, in-class instruction teaching specific 
job skills, such as word processing, electronics repair, and home health care; 

• on-the-job training, subsidized training that takes place as part of a paying job, 
often in a private sector finn (JTP A usually pays half of the wages for the training 
subsidy up to six months, but the jobs are supposed to be pennanent); 

• job search assistance, assessment of participants* job skills and interests, along 
with training in job-finding techniques and help in locating job openings; 

• basic education, including Adult Basic Education (ABE), high school or General 
Educational Development (GED, or high school equivalency) preparation, and 
Enghsh as a Second Language (ESL); 

• work experience, temporary entry-level jobs designed to provide basic employ- 
ment skills and instill effective work habits (the jobs may be subsidized by JTP A 
if they are in the pubic sector); and 

• miscellaneous services, including assessment, job-readiness training, custom- 
ized training, vocational exploration, job shadowing, and tryout employment, 
among a variety of other services. 

For adult and out-of-school youth **tcrminecs" who were enrolled in Title II-A programs 
nationwide during the sample intake period for the present study (November 1987 to 
September 1989), the most common specific services received were on-the-job training (28 
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percent of JTPA enrollees), classroom training in occupational skills (28 percent), and job 
search assistance (25 percent).' 



Participants 

Among the adults ar.d out-of-school youths who were enrolled in Title II-A nationally during 
the sample intake period for the present study, 95 percent were classified as economical h- 
disadvantaged.* About 86 percent were identified as facing one or more barriers to 
employment, including limited education, limited recent work experience, and others.^ 

The adults and out-of-school youths who enrolled in ITPA during this period were 54 
percent female and 46 percent male. In terms of their ethnic backgrounds, 54 percent were 
white, 30 percent were black, and 12 percent were Hispanic. About 65 percent were high 
school graduates; but 48 percent were receiving some form of public assistance when the\ 
applied to JTPA, and 29 percent were receiving Aid to Families with Dependent Children 
(AFDC). 



Performance 

One distinguishing feature of JTPA is its emphasis on program performance standards, 
especially with regard to the return on the program's investment in human capital, or the labor 
market skills and experience of program participants . For example, as stated in section 1 06(a) 
of the JTPA legislation: 

The Congress recognizes that job training is an investment in human capital 
and not an expense. In order to determine whether that investment has been 
productive, the Congress finds that — 



7. Job Training Quarterly Survey (JTQS) data. The JTQS is conducted by the U.S. Bureau of the Census, 
under conlracl to DOL, and reported by Westat, Inc. 

8. JTPA defines economically disadvantaged as having a family income equal to or below the poverty 
guideline set by the U.S. Ofiice of Management and Budget or 70 percent of the lower living standard set by 
the U. S. Department of Labor. The data presented in this and the following paragraph were computed from Job 
Training Quarterly Survey (JTQS) data for the relevant months. 

9. Ten types of barriers to employment were included: ( 1 ) having been employed 1 5 or fewer weeks dunng 
the 26 weeks before application to JTPA (67 percent of the enrollees); (2) lack of a high school diploma (35 
percent); (3) having reading skills below the seventh grade level (22 percent); (4) being an ex-offender (9 
percent); (5) being physically handicapped (9 percent); (6) being a war veteran (9 percent), (7) being a long- 
term AFDC recipient (9 percent); (8) being over 55 years old (6 percent); (9) having a limited English speaking 
ability (A percent); and (10) being a displaced homemakcr (3 percent). 
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(1) it is essential that criteria for measuring the return on this investment be 
developed; and 

(2) the basic return on the investment is to be measured by the increased 
employment and earnings of participants and the reductions in welfere 
dependency. 

As a result of this emphasis, DOL has expended considerable effort to develop a system 
of performance standards by which to judge SDAs' achievement of program goals,**^ The 
standards for adults focus on employment and wage rates, for participants in general and for 
welfare recipients in particular; those for youths focus on employment and attainment of one 
or more measures of skills enhancement. DOL also established standards for program costs, 
but less emphasis has been placed on those standards in the past several years. 

Among the adults who entered Title U-A during the sample intake period for the present 
study, 69 percent had entered an unsubsidized job before leaving the program (that is, before 
their cnroUment was terminated). The averagehourlywageforthosejobs was $5.86. Among 
out-of-school youths, 71 percent entered an unsubsidized job, began further training, or 
achieved another goal defined by DOL as a "positive termination."* * 

These standards measure certain outcomes of participating in JTP A Title II-A programs, 
but they provide no indication of program impacts. For example, the feet that 69 percent of 
adult terminecs found an tmsubsidizcd job docs not mean that JTPA caused their employment 
to occur. It is possible Aat all of these temiinees ^^iio found a job might have done so without 
access to JTPA; if this were true, then we would have to say the program had no impact. On 
the other hand, if very few adult terminees would have found a job without JTPA, then the 
program had a large impact. In other words, a program outcome measure alone docs not allow 
us to determine what the program actually caused to happen. 

To measure JTPA program impacts, one must compare the labor market outcomes of 
program participants with the outcomes they would have experienced without the program — 
as mWiSurcd by the experience of a control group whose members did not have access to the 
program. 



10. The original Title II-A performance standards measured only immediate post-program outcomes. DOL 
added several nveasures of subsequent labor market outcomes in program year 1988 

1 1 . The findings in this paragraph were computed from JTQS daU for a sample of JTPA terminecs who 
were enrolled in the program during the sample intake period for the present study. 
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Recent Policy Amendments 

Because the Job Training Partnership Act was enacted as permanent legislation, it has not 
been subject to periodic reauthorizations, as CETA was. The JTPA program has therefore 
had a more stable histor>' than its immediate predecessor, and was already a well-established 
program when this study began in 1987. 

In 1 986 Congress instituted minor changes in the program, and in 1 988 DOL esublished 
new performance standards. On September 7, 1992 President Bush signed the Job Training 
Reform Amendments of 1 992 into law (PL 1 02-367). These amendments to JTPA address the 
following issues, among others: 

Program targeting. In response to concerns that JTPAs emphasis on perfor- 
mance standards discourage SDAs from serving clients who are most in need, the 
amendments require that at least 65 percent of the adults and youths in the year- 
round program be persons with identifiable barriers to employment. 

Program services. The amendments require a formal objective assessment and 
an individual service sirateg}' for all program participants. Basic skills and 
occupational skills training must be provided if the assessment suggests they are 
needed and work experience or job search assistance may not be provided alone 
unless the assessment indicates this is appropriate. Furthermore, enrollment in 
OJT is limited to six months and this period must vary in accord with the level 
of skills for which training is provided. 

Program performance. The amendments specify that incentive grants to SDAs 
be based in part on the extent to which they serve persons with identifiable 
barriers to employment. In addition, performance standards must now reflect 
participants' acquisition of basic skills, achievement of specific occupational 
competencies or attainment of high school equivalency credential. 

Programs for youths. The amendments provide a separate title, Il-C, for year- 
round programs for youths. At least 50 percent of the participants in this new 
title must be out-of-school youths. The Summer Youth Emplo>Tnent and 
Training program. Title II-B, is maintained as a separate program. 

Other issues. The amendments also restructure current limitations on how 
SDAs can spend program funds, it modifies the basic formula for allocating 
JTPA ftmds to SDAs, it requires procedures to increase the fiscal accountability 
of SDAs, it specifies improvements to the data collected about local programs, 
and it includes provisions to enhance the coordination of JTPA programs with 
other human service programs. 

r-s 
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Previous Studies of Employment and Training Programs 

Researchers have been trying to measure the impacts of employment and training programs 
for as long as the programs have been part of federal social policy. Since the passage of 
MDTA in 1962, literally scores of these studies have been conducted. 

The Central Methodological Problem 

The central methodological problem in all of these studies has been how to determine vAiat the 
labor market experience of participants would have been without their access to the program 
in question. The most annmon approach has been to select a comparison group of persons 
as similarto program participants as possible, butwiio didnot participate in the program. The 
labor market outcomes of this comparison group have then been used to estimate what the 
participants' labor market outcomes would have been in the absence of the program. In 
addition, researchers have used statistical models to adjust their estimates for observed 
diflferences between participants (the treatment group) and the comparison group. 

The problem stems from the feet that the only way to adjust for differences between these 
two groups is by using individual characteristics that can be measured. Thus, one cannot 
control directly for characteristics that affect labor maricet outcomes but that cannot be 
measured fully, such as motivation. Nevertheless, if these unmeasured or partially measured 
factors are the same for program participants and comparison group members, on average, 
or if they correlate in specific ways with fectors that can be measured, they can be fiilly 
accounted for in estimates of program impacts. 

For example, if the motivation level of participants and comparison group members were 
the same, the effect of this fector on the labor maricct outcomes of each group would be the 
same. In this case the unmeasured characteristics of the two groups would balance out and 
would not bias the estimates of program impacts.'^ 

But if unmeasured characteristics that affect labor market outcomes are not well balanced 
between program participants and the comparison group, the impact estimates produced by 
comparison group methods will be biased. For example, if the motivation of program 
participants were higher than that of comparison group members, it would not be appropriate 
to attribute all of the subsequent difference between the earnings of these two groups to the 



12. Id addition, if the motivation level were difTerent for program participants and comparison group 
members, but if it were correlated in certain ways with characteristics that were measured (such as past earnings, 
age, gender, and race), then by controlling sutistically for differences in measured variabk:s one could 
simultaneously control for the differences in unmeasured factors. Once again, the effect of unmeasured 
characteristics would be '"neutralized." 
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program being studied. To do so would overstate the actual program impact, because even 
without the program, the participants would have earned more, on average, than the 
comparison group members. 

This problem of selection bias has been insurmountable for comparison group studies of 
the impacts of employment and training programs. Although a wide range of sophisticated 
statistical matching and modeling procedures have been used to address the problem, no 
acceptable solution has yet been reached. 

The basic limitation of the studies is simply that without perfect measures of the 
unmeasured variables, one cannot be certain vvdiether the selection bias has been removed! In 
£act, that certainty is possible only when the problem does not exist. Comparison group 
studies therefore require an assumption that the problem has been resolved by the procedures 
used to adjust for selection bias. But di£ferent procedures have produced different results; and 
we cannot choose among the procedures with confidence because we cannot know ^^ch 
procedures most successfully removed the selection bias. 

Random assignment is an alternative way to choose a group vdiose e^rience will reflect 
what program participants* labor market outcomes would have been without access to the 
program. Researchers are increasingly using this approach — ^which relies on a control group 
matched to the treatment group — ^because of its ability to eliminate selection bias. 

Basically, random assigtmient is like a lottery. Individuals first apply to a program and 
are screened to ensure their eligibility. Next, much like the flip of a coin, a computer randomly 
detennines who can enter the program and who caxmot. If there are more applicants than can 
be served by the program anyway, this procedure is a iai\x way to allocate the scarce resources 
involved. In addition, the laws of probability ensure that the applicants vAio are denied access 
to the program (the control group) do not differ systematically fi'om the applicants who are 
offered access (the treatment group) in any way, measurable or not. 

Thus, the subsequent labor market outcomes of control group members serve as valid 
estimates of what these outcomes would have been for treatment group members if the latter 
had not had access to the program. And therefore, the difference between the labor market 
outcomes of the treatment and control groups represents a valid estimate of the true impact 
of the program. 



Early Studies 

The numerous studies of employment and training programs conducted in the 1960s and 
1970s were generally limited to measuring short-term post-program earnings and employ- 
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ment, as well as a few demographic characteristics, for program participants and members of 
a comparison group. Differences in demographic characteristics between the treatment group 
ant! comparison group were controlled for using standard statistical methods (ordinary least 
squares regressions). Because the data and the statistical techniques used to control for 
selection bias in these studies were inadequate, litde systematic knowledge emerged from them 
(see Perry ctal., 1975). 



Second Generation Studies 

Several studies conducted later in the seventies and early eighties were based on longitudinal 
earnings data for program participants and con^arison group members (Ashenfelter, 1978; 
Kiefer, 1979; Cooley, McGuire, and Prescott, 1979; and Bloom, 1984b). These and 
subsequent studies applied relatively sophisticated statistical models to extorsive data on large 
samples." 

The basic s^proach was to adjust for differences in the prc-program earnings patterns of 
participants and comparison group members vAxcn comparing the post-^rogram earnings of 
the two groups. Here the assumption was that because prc-program earnings predict 
participants* post-program earnings without the program, ccmtroUing forthe difference inpre- 
program earnings between participants and compariscm group members would reduce 
selection bias to an acceptable level. 



The National CET A Evaluations 

Optimism in the research community about the ability of longitudinal earnings data to control 
statistically for treatment-control group differences in pre-prognun earnings and thereby yield 
valid program impact estimates led to the adoption of this second generation approach as the 
corestratcgy for the national CETAcvaluaticms, which began in the 1970s. The evaluations 
were based on data from the Continuous Longitudinal Manpower Survey (CLMS), the 
Current Population Survey (CPS), and earnings records maintained by the Social Security 
Administration (SSA). 

The CLMS was a large-scale survey of CETA participants. It collected detailed 
informaticHi on their individual characteristics and linked this information to annual earnings 
data on sample members in SSA records. The comparison group for the evaluations was 
drawn from the CPS. 



13. Ashenfelter (1978) used in autoregrcssivc model; Kicfa (1979), * fixcd<ffcct model; and Bloom 
(1984b), a timc-vmiying, fixcd<frcct model. 
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The CLMS and CPS data made it possible for researchers to combine statistical models 
of longitudinal earnings with a variety of procedures to match members of the comparison 
group to CETA participants, based on the detailed data on individual characteristics available 
for both groups.'^ 

Several major studies were commissioned to estimate CETA impacts from the CLMS. 
Exhibit 1 . 1 draws from Bamow*s (1987) detailed review of these studies. Most striking are 
the results for male participants, vMch ranged from estimates of small earnings gains to large 
earnings losses, depending on the study. But the results for females also varied substantially; 
three of the four studies found that CETA markedly increased annual earnings, but the fourth 
found almost no effect. Thus, for both males and females the estimates of CETA impacts 
depended critically on the statistical method used.^^ And according to Bamow (1987, 157): 

Data limitations and the iiud>ility to adequately test the validity of the 
selection processes assumed make it impossible to determine ^^ch studies 
modeled the process correctly. 

r 

Randomized Experiments 

In the mid* 1 970s researchers began to use an alternative approach, randomized experiments, 
to measure the impacts of employment and training programs . This approach, as noted earlier, 
employs a lottery to choose v^ch eligible applicants to a program are allowed to participate 
(the treatment group) and which are not (the control group). Again, the subsequent labor 
market outcomes of the control group serve as a valid estimate of vfoaX the outcomes of the 
treatment group would have been without the program; and thus, the treatment-control group 
difference in outcomes is a valid estimate of the program impact 

The first major employment and training study to use a randomized experiment was the 
Natiorud Supported Work Demonstration (Manpower DenKmstraticm Research Corporation, 
1980). Conducted between 1975 and 1979, tiie demonstration was a rigorous test of an 
intensive woric experience program for four groups: long-term AFDC recipients, young high 
school dropouts, ex-addicts, and ex<offenders. 



14. To select comparison group members, WcsUt (1984t) used discrete cell-matcfaing, and Dickinson, 
Johnson, and West (1984), a continuous Mahalanobis nearestHoeighbor matching procedure. Bassi (1983) and 
Bloom and Mclau^din ( 1 982) used a simple screening aitenoiL These authors alM 

their CETA evaluations, v^ch were reviewed in Bamow (1987). 

15. The ranges of impact estimates presented in Exhibit 1.1 for a givcpy i/m^ (row) reflect findings for 
different subgroups and thus are not shown here as evidence of a method-specific variation in impact findings. 
That evidence lies across the different studies, that is, in each column in the exhibit. 



12 • JTPA 18-MONTH IMPACTS / BACKGROUND 



Exhibii 1,1 Summary of EstimaiedCETA Impacts on Annual Earnings, from Four Studies Using 
the Continuous Longitudinal Manpower Survey (CLMS) 



Study (year published) Impact, in S 



a 


AduU women 


Adult men 


Bloom and McLaughlin (1982) 


800** to 1,300** 




200 


Dicldnton, Johnson, and 








Wot (1984) * 


13 




^90** 




White Minority 


WhUe 


Minority 




females females 


males 


males 


Bassi (1983) ^ 


740** to 778** 426** to 671** 


n/a 


117 to 211 


Wcsut, Inc. (1984) 


408** to 534** 336** to 762** 


-4 to 500** 


-104 to 658** 



A Sample meoben wete af et 25^; impacts were ior ptogram yean 1976-1978, coovected to 1980 doUacs. 
h. Sample membeti wett afet 22-64; impacu were (or proffam year 1978, repoctad ta aoauaal doUan. 

c. Sample mecobers were a^ea 23-60; impacu wece tot progtam yean 1977-1978, repoftad ia aoauaal doUan. 

d. Sample mecaben were a^ea 14-60; impacts were for ptof nm yean 1977-1978, reported ia nominal doUan. 
Statistically aitaificani at the .05 levd (two-tailed te«). 



The Supported Work Demonstration found large earnings impacts for AFDC recipients 
and small to negligible effects for the other three groups. But its successful use of a multisite, 
randomized experiment to measure the impacts of employment and training programs was an 
important finding in and of itself one that would set a methodological precedent for later 
research. 

As the desirability and feasibility of randomized experiments became more apparent, 
researchers began to use the approach more often. Several experimental studies of 
employment and training programs were initiated during the early and mid-19S0s; some are 
now completed, ^^e others are ongoing. 

Adults. Most of the studies of employment and training programs for adults focused on 
programs for welfu-e recipients,'^ although several others examined programs for displaced 
workers — persons who permanently lost well-paying, stable jobs because of foreign compe- 
tition or changing technology.'^ 

The largest randomized experimental study of employment and training programs to date 
is the Demonstration of State WorkAVelfiure Initiatives (Gueron and Pauly, 1991). Begun in 
1 982, this project tested a wide range of programs for welfare ^>p\ic9nXs and recipients in eight 
states, with a total experimental sample of over 45,000 persons. Some of the programs studied 
covered a broad cross-section of the AFDC caseload, were mandatory for AFDC recipients, 

16. See Gueron and Pauly (1991) for a comprehensive review of these studies. 

17. See Bloom (1990) and Corson et al. (1989). 
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and were operated as part of the existing Work Incentive program (WIN). Others covered onl\ 
selected portions of the AFDC caseload, were voluntar>', and were run as demonstrations to 
investigate the impacts of specific types of services. 

WorkAVelfare provided a wealth of information about the programs' administration, 
participation rates, costs, and impacts. The study also demonstrated the feasibility of 
implementing a rigorous randomized experiment at a verv large scale and in many sites 
simultaneously. 

Other major randomized experiments studying programs for welfare recipients include 
the AFDC Homemaker-Home Health Aide Demonstrations, conducted in seven states, with 
a 9,500-person research sample (Enns, Bell, and Flanagan, 1987); the Louisville WFN 
Laboratory Experiments, conducted at two local WIN offices, with a 4,200-person sample 
(Goldman, 1981); the Saturation Work Initiative Model (SWIM), conducted in San Diego, 
with a 4,600-person sample (Hamilton and Friedlander, 1989); and the already-mentioned 
Supported Work Demonstration, which had a component for welfare recipients in 10 sites, 
with a 1,400-person sample. 

From a detailed review of the findings from these studies, Gueron and Pauly (1991, 26) 
concluded that: 

Ahnost all of the \velfare-to-work programs studied led to earnings gains. 
This was true for both low- and higher-cost programs and services, and for 
broad -coverage and selective-voluntary programs ... .Seven of the nine broad- 
coverage programs led to increases in average aimual earnings, ranging from 
S268 to $658 in the last year of follow-up. Depending on the program, 
this was 1 1 to 43 percent above the annual earnings of people in the control 
group. The smaller-scale, selective-voluntary programs increased average 
annual earnings by $591 to $1,121 — 14 to 34 percent above the control 
group's earnings. 

The authors further concluded that: 

Earnings impacts for both low-cost job search and higher-cost programs 
were sustained for at least three years after program enrollment. 

Experimental studies of programs for displaced workers also provide a useftil point of 
reference, because they include adult men, who are not well represented in the other 
randomized experiments.^* Two studies are particularly relevant to the prer ent one: the Texas 

18. Exceptions arc the findings for ex-adt'icts and ex-convicts in the Supported Work Demonstration, 
almost all of whom were men, and the fmdings on Several WorkAVelfare programs that served men receiving 

AFDC-UP. 
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Worker Adjustment Demonstration (Bloom, 1 990), a three-site, 2,200-person study; and the 
New Jersey Unemployment Insurance Reemployment Demonstration Project (Corson et al 
1989), a io-site, 11,100-person study.'^ The programs, target groups, and economic 
environments examined were quite different in each, but both studies suggest that emplov - 
ment and training programs can increase the earnings and emploNment of displaced workers 
In addition, the Texas study indicates that program impacts were larger and more sustained 
for women tlian for men (Bloom, 1990, vii). 

Youths. The best existing information on the impacts of employment and training 
programs for youths is from the youth component of the National Supported Work 
Demonstration and the recent JOBSTART demonstration. 

As described earlier. Supported Work tested an intensive work experience program for 
four groups, one of which was a group of about 900 young high school dropouts, most of 
whom were male, located in five sites. Findings from the study indicated negligible impacts 
on post-program earnings or employment for those youths (Manpower Demonstration 
Research Corporation, 1980). 

JOBSTART is an ongoing study of 2,200 young high school dropouts in 13 sites. 
Interim findings suggest a minimal post-program impact on the earnings of female youths 
and a negative impact on male youths. Specifically, during the second year after random 
assignment the impact on young men was $-667, or 1 3 percent less than what they wou Id have 
earned without access to the program. Additional follow-up is under way to determine 
whether this negative impact persisted for male youths and whether the impact for female 
youths rose or declined over time. 



Methodological Studies of Experimental and Comparison Group Techniques 

While the findings from the national CETA evaluations were becoming available, a series 
of studies was conducted to examine the methodological properties of experimental methods 
and nonexperimental comparison group methods. Fraker and Maynard ( 1 984) and LaLondc 



19. The Buffalo Dislocated Worker Demonstration Program (Corson. Long, and Maynard, 1 985) is another 
example. Although conducted as a randomized experiment, the study estimated program impacts using 
nonexperimental comparison group methods. Thus, its fmdingsare not directly comparable to those of the Texas 
and New Jersey demonstrations. 
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(1984) used data from the Supported Woric Demonstration to test the ability of comparison 
group methods to emulate the findings from the e?q)eriment. Ashcnfclter and Card (1985) 
used CLMS data to explore the variation in findings from different comparison group 
methods ^plied to the same data. The three studies reached the same basic conclusion. As 
Fraker and Maynard reported (1987, 216, 220): 

The overvviielming conclusion from this study is that comparison group 
study designs should be avoided \vhen reliable estimates of program impacts 
are an important study objective.... 

For the time being the safest evaluation strategy involves the use of a true 
control group. 

Recommendations of the National Academy of Sciences 
Committee on Youth Employment Programs 

In 1985 the National Academy of Sciences convened a committee to review the existing 
research on employment and training programs for youths, especially those frmded through 
the Youth Employment Demonstration Projects Act (YEDPA). The committee found that 
little could be said with confidence about the impacts of programs for youths, because the 
comparison group strategies that had been used to study theprpgrams did not offer convincing 
evidence. TheconMnittecalsoconcludedthat(Bctsey,Hollister,andPapagcorgiou, 1985, 18, 
30): 

control groups created by random assignment yield research findings about 
employment and trainingprograms that are far less biased than results based 
on any other method.... 

Future advances in field research on the efficacy of employment and training 
programs will require a more conscious commitment to research strategies 
using random assignment. 

Recommendations of the JTLS Research Advisory Panel 

Soon after JTPA was authorized in 1 982, the Department of Labor began plans for a national 
evaluation of the program. This evaluation was tobuild on the longitudinal comparismgroup 
approach used in the CETA evaluations. It was to include a detailed survey for a national 
sample of JTPA participants, referred to as the Job Training Longitudinal Survey (JTLS) and 
a special national survey, the Survey of History of Work (SHOW) for constructing a 
comparison group (Westat, 1984b). 
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But when the inconsistent findings from the various CETA studies began to emerge, and 
some of the early findings £rom the methodological studies of experimental and comparison 
group techniques were becoming available, DOL staff members began to rethink the 
Department's plans. Seeking guidance on this issue, DOL convened a panel of experts; and 
authors of the CETA studies were invited to present their findings and recommendations to 
the panel. The panel concluded (Stromsdorfer et al., 1985, 21): 

The recommendations of the panel are strongly conditioned by the judgment 
that it will not be possible to solve the problem of selection bias within the 
context of a quasi-e}q>erimental design such as the JTLS/ SHOW, at least not 
in a short enough time fiame to meet Congress' needs for valid information 
to guide policy. Even though many authors studyilng employment and 
training programs have recognized the selection problem, no such study 
using a quasi-experimental design can be said to have controlled adequa t ely 
for selection bias. The panel docs not intend to set forth a counsel of despair. 
Rather, it is concerned that the past cvaluaticms of CETA have consumed, 
and ^heconten^)lated evaluations of JTPA will continueto ccmsume, millions 
of dollars and much valuable time. It would be extremely unfortunate if the 
analysis of the JTLS/SHOW design would yield ihc same ambiguous 
conclusions as has the analysis of the CLMS/CPS database for CETA. 

There were also well-acknowledged trade-ofiEs with the alternative: a nmdomized 
experiment. On the one hand, the panelists understood that the c}q)erimental approach 
represented the best chance to obtain valid and reliable impact estimates for the local programs 
tobestudied. On the other hand, they rccc>gni7f)d that rK>t all local programs would agree to 
participate in such a study, and thus it would be difficult to obtain a probability sample of sites 
to ensure the generalizability of fiiuiings to the JTPA program naticxially. 

On balance, then, the advisory group decided that without valid estimates for the sites in 
the study, the issue of generalizability was not relevant. Its recommendation was Aerefore (p. 

22): 

The DOL should perform a selected set of classical experiments over the next 
several years that i; olve random assignment of prpgram-eligible individu- 
als to the treatment (experimental) group and to the non-treatment (control) 
group. This isthe key recommendation ofthepanel. The intent is to use these 
experiments to: 



evaluate the net impact of JTPA for selected target/ 
treatment groups in a set of SDAs that volunteer to 
participate. 
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• use these experimental results and the understanding 
of the selection process gained thereby to improve 
the effectiveness of quasi-experimental designs as 
a strategy for program evaluation. 

The National JTPA Study in Brief 

In June 1986 DOL awarded two separate contracts to conduct the National JTPA Study: 

• a Part A contract with the Manpower Demonstration Research Corporation 
(MDRC) and its subcontractors, the National Governors' Association, the 
National Association of Counties, and the National Alliance of Business, to 
implement and monitor the experiment; and 

• aParr5contractwithAbt Associateslnc.ar.'^its subcontractors. New York 
University, MDRC, the National Opinion Research Center (NORC), Fu 
Associates, and ICF, Inc., to design ^ study, collect the required data, and 
conduct the analyses. 

E.^ed on the recommendations of the JTLS Research Advisory Panel, the National JTPA 
Study consists of two parts: 

• a randomized experimental study of JTPA Title II-A programs, which is 
based on the experiences of 20,601 eligible adults and out-of-school youths 
yAiO applied to 16 local SDAs in the continental United States between 
November 1987 and September 1989; and 

• a nonexperimental methods study to develop new comparison group proce- 
dures for estimating program impacts. 

The core of the study is the randomized experiment, in which eligible program applicants were 
randomly assigned to either a treatment group, whose members were offered access to Title 
n*A services, or a control group, w^se members could not obtain those services for a period 
of 1 8 months. (The control group could, however, obtain employment and training services 
from other local programs.) As demonstrated in Bloom (1991) and Appendix A here, the 
treatment and control gioups were indeed well matched, as one would expect from a strictly 
applied random assignment procedure. 
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As noted in the introduction to this ch^ter, because of the large sample size (20,601), 
the study is able to make valid treatment-control group comparisons for a variety of different 
subgroups, including four main target groups, groups recommended for different clusters of 
services, and selected key subgroups of interest to policymakers and program planners. 

The primary goal of the National JTPA Study— to estimate the effectiveness of 
Title II-Aprograms as they normally operate — called for certain key decisions on the study's 
design.^ 

First was the challenge of recruiting and selecting SDAs to sc tc as sites. Because the 
study did not have a legislative mandate that required SDA participation, it had to rely on 
SDAs that were willing to volunteer. But SDAs were concerned about participating in the 
study for a number of reasons and hence were reluctant to participate. 

For example, the experimental design required to address the key research questions of 
the study was complex, and SDAs were concerned about its possible effects on their 
programs. Inaddition, SDAs were concerned aboutthe potential politicalfidloutthatrandom 
assignment might generate. Further complicating matters was the &ct that in order for an 
SDA to participate, all local organizations and key individuals involved (the SDA, its PIC, 
the vendors, and local government officials) had to agree. This requirement of unanimity 
greatly reduced the chances that a prospective site would volunteer. 

For these reasons, it was not possible to draw a strict probability sample of sites.^' 
Instead, a range of SDAs from across the country were recruited to participate. And from 
among them, the 1 6 SDAs that were willing and able to participate became sites for the study 
(see Doolittle and Traeger, 1990). 

Second, because JTPA program staff often reconnmend more than one program service 
for an applicant, the study was designed to measure the impacts of clusters of program 
services — ^what we term service strategies~xhot single services in isolati(Hi, such as class- 
room training in occupational skills, or on-the-job training, or job search assistance. Isolating 
the impacts of single services would require comparing the experiences of treatment and 
control group members for each. But to construct such treatment and ccmtrol groups would 
require a special demonstration that would have to be nm quite differently from JTPA 
programs. 



20. See Bloom et al. (1990) and DooUtUe und Trueger (1990). 

2 1 . Original plans called for a probability sample of sites, although the difSculty of achieving this objective 
was acknowledged from the outset. When it became clear that this approach was not feasible, given the 
constrmints of the present study, the process was modified to ooe that focused on recniiting the most diverse 
^oup possible of SDAs that ^ctc willing and able to participate. 
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Instead, this study was designed to estimate the impaci s of three distinct service strategies : 
one that recommended sample members for classroom trailing in occupational skills (and in 
some cases other, secondary services); a second that recommended sample members for on- 
the-job training (and in some cases other, secondary services); and a third that recommended 
sample members mainly for other services brides classroom t»iaiaing in occupational skills 
and OJT. The mix of services sample members actually received was distinctly different for 
each services strategy and reflected in part (but not entirely) the main service for which the 
sample members were recommended. 

Third, because control group members would be able to receive employment and training 
services from other, non-JTPA providers, the study was designed to estimate the effect of 
JTPA as an incremental source of these services. This is probably the most relevant 
comparison to make, because JTPA expenditures add services to the existing landscape of 
employment and training programs. Thus, to assess the program in this regard requires 
examining the extent to which JTPA adds services to the local community and, in turn, the 
extent to which this increment in services resulted in an impact on labor market outcomes for 
the treatment group. Our forthcoming final report v;ill also compare the incremental costs of 
adding JTPA services and the incremental impacts of the program, to determine the cost- 
effectiveness of the local programs s^dicd. 

Finally, because local JTPA program staff can only offer program services to applicants, 
not force them to enroll, the study is designed to provide estimates of the impact of offering 
access to JTPA services, not the impact of receiving them. From these estimates it is also 
possible, however, to infer what the likely impact of receiving program services was. The 
study will provide these inferred estimates as an additional perspective from which to judge 
the program's effectiveness; but less confidence can be placed in these estimates because they 
are inferred indirectly. 



Summary 

The National JTPA Study offers important substantive and methodological contributions to 
the literature, especially in light of how little is known about the effectiveness of employment 
and training programs, and bow to measure their effectiveness. 

The study will provide valid and reliable evidence on the effectiveness of JTPA Title 
II-A programs in a diverse group of sites . It will identify whom those programs are working 
for (or not) and which services strategies are working (or not) for each target group. By 
identifying program successes, the study can help guide future efforts to study the frtctors that 
promote success. And by identifying situations in which the program is not working, the study 
can help target efforts for change. 
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But the study findings by themselves cannot provide a blueprint for action. They can only 
identify issues to be addressed in the future, vAnch must, in turn, be based on the development 
and rigorous testing of new approaches to serving the labor market needs of disadvantaged 
persons. 
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Preview: The 18-Month Impact Analysis 



Ti HIS chzptCT describes the 1 Smooth impaa analysis of the National JTP A Study. The 
first main section outlines the in^lementation of the experimratal design, indicating how 
the 1 6 study sites were selected and how client intake and random assignment were conducted. 
This section also defines the four target groups and three service strategy subgroups for vMch 
program impacts were estimated. 

The second main seaion describes the types of program in^acts estimated in the 18- 
month analysis. The section b^;ins by defining the 18-mon& study sanq>le and then 
distinguishes between impacts on the treatment group overall (impacts per JTP A assignee), 
\diich were estimated directly fi-om the experimental data, and impacts on those treatment 
group members who were actually enrolled in the program (impacts per JTP A enroUee), 
which were inferred using a simple extension of the e>q>erimental data. We then e?q)lain how 
the impact estimates rqpresent the impact of the increment in employment and training 
serv'ices that Title II*A provider , beyond those otherwiseavailable to low-income Americans. 
The section ends by defining the educaticml attainment and labor maricet outcomes used as 
the basis for measuring program impacts in this report.^ 

The last section describes the five sources of data employed in this report: a Background 
Information Form completed by sample members vfbca they applied to JTP A; our First 
Follow-up Survey; enrollment and tracking data from the 16 sites; quarterly earnings data 
fiom state unemployment insurance agencies; and the Job Training Quarterly Survey of JTPA 
participants nationwide. 

1 . Our forthcoming final rqx)rt will provide estimates of these impicts over « longer follow-up period, as 
well MS estimated impacts on welfare receipt and a benefit-cost analysis. 

21 
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Implementation of the Study Design 

As explained in Chapter 1, the primary goal of the National JTPA Study is to provide valid 
estimates of the impacts of JTPA Title II-A programs as they were being run at the time of 
the study in a broad range of different sites. To produce valid impact estimates, the study was 
implemented as a classical randomized c)q)eriment, with matched treatment and control 
groups. 

This section briefly describes the selection of sites for the study and then describes the 
random assignment process that created the treatment and control groups. We also explain 
our focus on four main target groups, defined by age and gender, and three main service 
strategies, defined by the specific program services SDA staff recommended for members of 
the study sample. Because of the size of the experimental sanq)le for this 1 8-month analysis 
(17,026) and the nature of the randcwn assignment, we have in essence separate experiments 
for each of these main subgroups, as well as for smaller key subgroups defined by such 
characteristics as ethnicity and the barriers to employment sample members were facing when 
they applied to JTPA. 



Site Selection 

As noted in Chapter 1, the 16 study sites were recruited fi-om among service delivery areas 
(SDAs) in the continental United States.^ As described in Chapter 3, and in Doolittle 
(forthcoming), these SDAs represent abroad range of different administrative arrangements, 
program services, participant characteristics, and labor market conditions. The program 
impacts reported here therefore reflect much of the diversity that exists within JTPA 
nationwide. 

The sites do not, however, represent a probability sample of SDAs that would allow us 
to generalize the study findings to the Title II-A program nationwide. For reasons detailed in 
the final design report (Bloom, Orr, Doolittle, Hotz and Bamow, 1990) and in the first 
implementation report (Doolittle and Tracger, 1990), it was not possible to recruit such a 
sample. Instead, we recruited SDAs based on the following criteria: their diversity, their 
willingness to participate, their ability to implement the experimental design, the size of the 
experimental sample they could provide, and the likely c(Mnposition of this sample. 

Diversity was a key criterion because of our desire to provide estimates of program 
impacts under as broad a range of conditions as possible. We did not want to base the study 
on a few isolated SDAs that were similar to one another and different fi'om most others . And 

2. In JTPA parlance "service delivery area'* refers to both the local administrative agency for the program 
and the geographic area it serves. 
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we particularly did not want ^sample of SDAs that were unusually successful or unusually 
unsuccessful in tenns of the JTPA standards used to assess their perfoimance. 

SDAs* willingness to participate was essential because there was no legislative mandate 
that required them to do so. Moreover, administrators' concerns about problems that might 
arise if they did were a major obstacle to overccmie. One problem that was especially acute 
was that all of the parties afiGIiated with the SD A (the SDA director and staff, members of the 
Private Industiy Council, local service providers, and local government officials) had to agree 
before it could enter the study and properly implement study procedures. 

SDAs' abUity to implement the &irly complex experimental design, without unduly 
disrupting their normal operations, was also essential. We therefore did not recruit some 
SDAs that might have b^ willing to join the study but were e^eriencing administrative 
difficulties. 

The size of the experimental sample each SDA could provide was another important 
consideration. Not only did we need a large total experimental sample, but we also had to limit 
the number of sites, for logistical reasons. We therefore did not recruit sites with fewer than 
500 Title n-A terminecs in program year 1984 (the most recent year for which data were 
available at the time). 

Finally, we also took into account the composition of the potential sample at each SDA, 
to help insure that the sample represented a broad mix of program participants. 

The preceding criteria were not embodied in formal site selection rules. Instead, they 
served as an ii^formal guide to help direct the marketing and outreach efforts of the 
implementation team. 

Exhibit 2. 1 shows the names and locations of the 16 SDAs that ultimately participated 
as sites in tk:^ study. In each SDA the experinient included virtually all of the eligible adults 
and out-of-school youths who applied to JTPA Title II*A during the sample intake period for 
that SDA and who were judged by SDA staff to be 2q)propriate for program services.^ That 
period differed for each SDA, but the first sample member entered the study inNovember 1987 
and the last one entered in September 1989. 



3. Fonnal agreements with some of the SDAs excluded certjin snudl groq>s of applicants from the study 
(and therefore from ruuiom assignment) for one of three main reasons: ( 1 ) logistical problems, such as widely 
dispersed groq>s that would have required many different intake locations; (2) recniitment problems for 
particular groups, such as older workers; and (3 ) the non voluntaiy nature of certain i^lications, luunely, among 
groups required to apply to JTPA either by the courts (usually as a condition for parole) or as a condition for 
receiving public assistance. Doolittle and Traeger (1990) describe the groups that were excluded from the 
experiment, if any, at each site. 
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Exhibit 2. 1 Location of the 16 Study Sites 




Key: 

1. Fort Wayne, Indiana 

2. Coosa Valley, Georgia 

3. Corpus Christi, Texas 

4. Jackson, Mississippi 

5. Providence, Rhode Island 

6. Springfield, Missouri 

7. Jersey City, New Jersey 

8. Marion, Ohio 



9. Oakland, California 

10. Omaha, Nebraska 

1 1 . Larimer County, Colorado 

12. Heartland, Florida 

13. Northwest Miimesota 

14. Butte, Montana 

15. Decatur, Illinois 

16. Cedar Riq>ids, Iowa 



Notes: The ttudy fite* are lifted in descending order by the liie of the 18-month itudy lample •! each site. For the full name of 
each •crvice delivery area, iu largest city » and tu sample sixc, sec Exhibit 3.1 in Chapter 3. 
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Client Intajce and Random Assignment 

Exhibit 22 illustrates the client intake and random assignment process used to create the 
treatment and control groups. Although specific details of the process varied from site to site, 
to accommodate existing local Listi;utional arrangements and operating practices, the same 
basic procedure was followed at all sites. 

The process began with normal JTPA procedures for recruiting applicants and determin- 
ing their eligibility for the program/ Those applicants who were judged to be eligible were 
then assessed by local SDA staff members to determine which JTPA services would be most 
appropriate for meeting their individual needs. At that point the staff members recommended 
applicants for one or more specific program services,^ Those recommendations, in turn, 
formed the basis for assigning all sample members to one of three service strateg\' subgroups: 
classroom training, OJT/JSA, and other services (defined in a later subsection). 

During this process staff members explained to applicants that not all of them could be 
served, and because the SDA was participating in a special study, a lottery would be ust J to 
select those who would be allowed to participate in JTPA and those who would not over the 
next 1 8 months. Applicants then signed a consent form to indicate that they understood the 
nature of the participant selection process and to allow the research contractor to obuin 
information on their earnings, employment, and welfare receipt from the administrative 
records of governmental agencies. 

At that point an SDA staff member telephoned a random assignment clerk from the stud>' 
team, who randomly assigned each applicant to treatment or control status within each ser\'ice 
strategy subgroup, as shown in Exhibit 2.2. Specifically, two-thirds of the experim; ntal 
sample was assigned to the treatment group (whose members were allowed to receive JTPA 
Title II-A services), and one-third was assigned to the control group (whose members were 
not allowed to receive those services for the experiment's embargo period of 18 months).^ 



4 . Sites were given a limited amount of technical assistance to improve their client recruitment procedures, 
so that enough eligible people would apply to JTPA to provide for a control group without reducing the number 
of persons served by the SDA. To the extent that this additional recruitment changed the mix of clients in the 
programs, and to the extent that any such change in client mix produced a change in average impacts, the 
additional recruitment may have altered the nature of the population for which the impact findings can be 
generalized. There is no empirical evidence with which to assess this possibility, however. 

5. Throughout this report we refer to classroom training in occupational skills, basic education, on-the-job 
training, job scare* . assistance^ -vork experience, and miscellaneous services as specific program services. They 
are often referrec» to as program activities in the employment and training literature. 

6. This 2/1 ratio of treatment group members to control group members represents an explicit trade-off 
between the need for statistical precision in program impact estimates (the optimum ratio for which is 1 / I ) and 
a practical need to minimize the size of the control group in order to minimize the number of persons that had 
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Exhibit 2. 2 Random Assignment Model for the National JTPA Study 
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Tteatment 
Group 



Control 
Group 
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The SDA staflFthen called or wrote treatment group members to schedule their partici- 
pation in JTPA; control group members usually were informed of their status by letter, 
although some were informed by telephone or in person. 

The Four Target Groups 

From the outset the study was designed to focus on three main target groups: adult women, 
adult men, and out-of-school youths. Because members of each target group were randomly 
assigned to treatment or control status independently of one anotL^r, the random assignment 
process produced an independent randomized e>q>eriment for each of these target groups. 

The distinction between adult women and men is based on the accumulated evidence of 
differences between the impacts of employment and training programs estimated for the two 
groups (for example, Ashcnfeltcr, 1978;Kiefcr, 1979;Bassi, 1983; Bloom and McLaughUn, 
1982: Wesut, 1984a; and Dickinson, Johnson, and West, 1984). Out-of-school youths were 
separated from adults in the analysis because c /the major dififcrences between their positions 
in the labor market. In addition, findings for out-of-school youths are reported here separately 
for female youths and male youths because the observed impacts appear to differ by gender 
(see Chapter 6).' 

The study was limited to out-of-school youths in Title II-A rather than all Title II-A 
youths, because the evaluation team e}q)ected that programs atui relevant outcomes for in- 
school youths and out-of-school youths would differ too much for them to be analyzed together 
and because the samples for in-school youths were expected to be too small for separate 
analyses. We also anticipated that it would be difficult to obtain consent to implement random 
assignment in public schools, where some JTPA services for in-school youths are provided. 

In the JTPA Title II-A program nationwide 30 percent of the participants are adult 
women, 25 percent are adult men, 23 percent are out-of-school youths, and 22 percent are in- 
school youths. The National JTPA Study therefore focuses on target groups that make up 
about three quarters of the population currently being served by JTPA. 



to be turned away by local prognin stafTand the number of additional applicants that had to be recniited to 
provide for a control group. 



7. Original plans were for impacts on youths to be analyzed separately for white and minority youths. But 
estimated impacts dilTered less between white and minority youths than between female and male youths (see 
Chapter 6). Hence, the impact analysis for youths focuses on this latter distinction. 
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The Three Service Strategies 

The program services that JTPA applicants ultimately receive depend on a number of factors, 
including the types of services that the applicants want, judgments by program staff about the 
suitability of specific services for particular applicants, and the availability of certain services 
at the time an individual applies to the program. The specific program service or services 
provided to an applicant are some^unes determined deliberate planning (for example, a 
basic education course followed by occupational skills training). At other times, however, 
they are determined by trial and error, producing a sequence of services that evolves from a 
continuing effort to find one or more that are suitable. 

In short, it is difficult to predict which service or services an applicant will receive. And 
JTPA often provides more than one service to an applicant. As a result, it was not possible 
both to achieve our mandate to examine the impact of JTPA programs as they were being 
operated at the time (Bloom, Orr, Doolittle, Hotz, and Bamow, 1990) and to isolate the effect 
of receiving a particular program service, because to isolate the effect of a single specific 
program service would require restricting certain sample members to that service, substan- 
tially altering the normal decision-making process of JTPA. 

Thus , to examine the impacts of the different types of services offered by JTPA programs, 
we grouped treatment and control group members into three service strategy subgroups 
defined in terms of the specific program services recommended for them before random 
assignment.* We based our definitions on service recommendations because (1) doing so 
made it possible to match treatment group members and their control group counterparts 
(which would not have been possible using program services received); (2) we judged that 
service recommendations were the best available predictors of services received and therefore 
the best available way to distinguish among sample members according to the services the\' 
did subsequently receive; (3) this approach had a minimal effect on the normal JTPA decision- 
making process, and (4) it niade it possible to account for the combinations and sequences of 
services received by many JTPA participants. 

Exhibit 2.3 shows the services allowed in the definition of each service strategy. The 
definitions are based on the initial staff recommendation for each sample member, with modest 



8. Previous reports for this project (Bloom, Orr, Doolittle. Holz, and Bamow, 1 990; Doolittle and Traeger. 
1 990; and Bloom. 1 991 ) icnned the service strategies **trcalmcnt streams" and termed the three strategies as 
classroom training, on-the-job training, and other activities. The names and characterizations of these service 
strategies evolved over time as we learned more about the actual services received by each service strategy 
subgroup. 
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Exhibit 2.5 Specific Program Services Allowed in Each of the Three 
Service Strategy Definitions 



Service strategy 

Specific program Classroom OJT/ Other 

service training JSA services 



Classroom training in 



occupational skills 


Yes 


No 


Yes 


On-the-job training 


No 


Yes 


Yes 


Job search assistance 


Yes 


Yes 


Yes 


Basic education 


Yes 


Yes 


Yes 


Work experience 


Yes 


Yes 


Yes 


Miscellaneous 


Yes 


Yes 


Yes 



restrictions on the specific program services that participants could subsequently receive. 
Specifically:^ 

• The classroom training strategy was defined to include sample members who 
were recommended for classroom occupational skills training but not for on-the- 
job training (OJT). Any other service — such as job search assistance, basic 
education, and work experience (but not OJT) — could be recommended in 
addition to the defining service for this strategy. Most sample members recom- 
mended for this service strategy who were subsequently enrolled in JTPA 
received classroom training in occupational skills or basic education or both (see 
Exhibit 3.19 in Chapter 3). 

• The OJT/JSA strategy was defined to include sample members who were 
recommended for OJT but not classroom occupational skills training. All 
secondar>' services (but not classroom occupational skills training) could be 
recommended in addition to the defining service for this strateg\\ Most sample 
members recommended for this service strategy who were later enrolled in JTPA 
received OJT or job search assistance, or both (see Exhibit 3.19). 



9. Two infrequent exceptions to the service su^legy definitions presented here were limited classroom 
training provided to some members of the OJT/JSA subgroup before lhc\- received on-the-job training and 
limited on-the-job training provided aClcr some members of the classroom training subgroup received classroom 
iraming in occupational skills. 



Hi 
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• The other services strategy was defined to include sample members who were 
recommended for neither classroom occupational skills training nor OJT as the 
defining service.'^ This strategy produced a substantially different mix of 
services for adults than for youths. Adults recommended for this strategy who 
were later enrolled in JTPA received mainly job search assistance and miscella- 
neous services, such as customized combinations of classroom occupational 
skills training and OJT (sec Exhibit 3. 19). Youths recommended for the strategy 
who became enrolled in JTT A received mainly basic education or miscellaneous 
services, such as tryout employment (in which participants are hired on a 
probationary basis to learn a job and prove themselves qualified for permanent 
employment) and job shadowing (in which participants follow and observe a 
regular employecto learn whatis required to holdajob: sceExhibit3.19). Hence, 
for adults this strategy focused more on unmediate employment, whereas for 
youths it focused more on educaticm and entry-level job skills. 

As Ch^ter 3 will demonstrate, these definitions produced subgroups that did, in fact, 
receive distinctly different sets of prpgram services. Note, however, that each service strategy 
subgroup ultimately received predominantly two key services. Hence, the study findings 
reflect more tha n the impact of the single defining service for each strategy. 



Impact Estimates in the 18-Month Analysis 

This section briefly describes how we obtained the inqjact estinutcs presented in this report, 
detailing, first, the 1 8-month study sample upon which these findings are based; then, how we 
estimated impacts per JTPA assignee (treatment group member) and impacts per JTPA 
enrollee (treatment group member who actually enrolled in the program); and finally, how we 
measured the attainment of a training elated high sdiool diploma, earnings, employment, and 
the components of earnings — ^the outcomes of interest in this analysis. 

The 18-MomTi Study Sample 

The random assignment process described above produced a total e5q)crimcntal sample of 
20,601 treatment group and control group members from the 16 study sites. Bloom (1991) 
describes the baseline characteristics of this sample.^' 

10. Foraial igrconcnts were midc with each SDA to specify a maximum allowable percentage of 
experimental unxpic members recommended for the other services strategy. Ttis limit was based on the 
previous experience of each site, and no site reached its limit 

1 1 . Bloom (1991) describes an experimental sample conuining 20,602 cases. Subaequcntly, two of these 
cases were discovered to represent the same person. 
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Exhibit 2.4 Sample Sizes in the 18-Month Study: Full Sample and Target 
Groups, by Service Strategy Subgroup 





Full 


AduU 


Adult 


Female 


Male 


Service strategy 


sample 


women 


men 


youths' 


youths" 


subgroup 


(1) 


a) 


(3) 


(4) 


(5) 


Classroom training 


6.113 


2.927 


U53 


1,193 


640 


OJT/JSA 


6.410 


2322 


2,754 


612 


722 


Other services 


4^03 


13S8 


1,519 


844 


782 


All subgroups 


17,026 


6,607 


5,626 


2,649 


2.144 



Source: Unadjufted frequencies bued on Background Infomwtion Foim ittpontet. 
A Out-of-achool youths only. 



The present report on 18-montfa impacts is based on a 17,026^rson subsample that 
includes all members of the experimental sample who were sdieduled for a First Follow-up 
Survey interview (described later in this chapter) at least 18 months after their random 
assignment. We refer to this subsample as the IS-month study sample. 

Because sample members' scheduled interview dates were independent of >^etber they 
were c^signed to ^e treatment group or the control group, the 1 8<month study sample is fully 
experimental. Hence, the treatment group and the control group for this sample had no 
systematic differences at random assignment, and their measured baseline characteristics 
were virtually identical as shown in Appendix A. 

Exhibit 2.4 shows the size of each target group and service strategy subgroup in the 18* 
month study sample. By target group the sample includes 6,607 adutt women, 5,626 adult 
men, 2,649 f male youths, and 2, 144 male youths. By service strategy subgroup it includes 
6,113 persons recommended for the classroom training strategy; 6,4 10 persons recommended 
for the OJT/JSA strategy, and4,S03pers(xis recommended forthe other services strategy. For 
e?q)erimental purposes we therefore have large samples for each target group and for each 
service strategy subgroup . For adult women and men the sample sizes for the service strategy 
subgroups were also quite large, ranging from 1,353 to 2,927 persons; for female and male 
youths these subgroup sample sizes were smaller but still substantial, ranging from 612 to 
1,193 persons. 

12. In Ove sites that experienced recruitment problems the treatmeat^ontrol group ntio >vis izmased 
temporarily from 2/1 to 3/1 or 6/1 . This reduced the number of eligible applicants lost to the pfx>gram because 
they were assigned to coDtrol group status. Consequently, the overall treatment<oatrol groq) ratio for the full 
experimental sample is slightly greater than 2/1. When constructing the 1 S-month study sample, however, wc 
nndomly deleted these "extra" treatment group merobcra, thus producing an analysis aan^>le with a constant 
2/1 ratio for all sites and subgroups. This was done to simplify the cofTeq)onding estimates of program impacts, 
as discussed in Appendix D. 
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Although the 18-month analysis is intended to represent impacts on the full 18-month 
study sample of 17,026 persons, the impact estimates reported were obtained from foUow-up 
data for a subsample of 14,94 1 persons (88 percent of the 18-month study sample) because 
the required follow-up data wereavailableonlyforthissubsan^le(seeAppendixD)." Hence, 

the impact findings for each of the subgroups discussed above, and for all other subgroups 
discussed in this report, are based on foUow-up data for most but not all of their counterparts 
in the 18-month study sample. 



Impacts per JTPA Assignee 

Because the random assignment process outlined in Exhibit 2.2 produced treatment groups 
and control groups withno systematic diflfercnccs at random assignment, the subsequent labor 
market experience of the control group provides a valid estimate of what the experience of the 
treatment group would have been if JTPAhadnot beenavailable to its members. For example, 
if the mean earnings of the control group were $7,000 during the first year after random 
assignment, one could infer that the treatment group would have earned this amount (plus or 
minus a margin to reflect random sampling error) in that same year without assistance from 
JTPA. 

Moreover, if the actual mean earnings of the treatment group were $7,500 during that 
year, <»e could infer that JTPA increased treatment group earnings by $7,500 minus 
$7,000, or $500, on average (plus or minus a margin to reflect randtxn ssinxtphng error). 

Similar logic can be used to estimate program impacts on dichotomous outcomes that are 
naturally expressed in percentage terms. For example, if 80 percent of the treatment group 
were employed at some time during the first year after random assignment, and 70 percent of 
the control group were employed during that time, the best estimate would be that JTPA 
increased employment by 10 percentage points. 

These impact estimates rely exclusively on direct comparisons of outcomes for all 
ueatment group members (whether they were subsequently enrolled in JTPA or not) and all 
control group members. Hence, they rq)rcscnt the average impact of the program on all 
sample members who were randomly assigned to the group having access to the program: the 
ueatment group. We refer to ihese findings as estimar.i of impacts per assignee, and they 



1 3 At least 1 8 months of usiblc follow-<qj data were obtained firom theFint FoUowmp Survey for 14,446 
of these sample members. In addiUoo. to adjust for survey nooresponse bias in the impact estimates for adult 
vramen (the only target git«^ for ¥*ich this wu judged to be a problem), we imputed foUow-up data from 
earnings daU flwn sUte unemployment insurance agencies for as many survey noorespaodenU as possible 
(495) Thus for the imp^-ct analysis mw used foUowmp data for 14,941 persons, including 98.0 percent of the 
adult women in the 18-mooth study sample. 78.5 percent of the adult men, 86.8 percent of the female youths, 
and S 1 .5 percent of the male youths. 
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represent the effect of providing treatment group members with access to JTPA Title II-A 
services relative to what they could have accomplished without access to those services. 

This comparison of treatment and control group outcomes can be conducted separately 
for many different subgroups within the ISnnonth study sample, thereby providing separate 
program impact estimates for each subgroup. In effect, the experimental design allows for 
a separate experimental treatment-control group comparison for any sample subgroup that 
can be defin^ in terms of common fectors measured before or at random assignment. For 
example, the design can yield sq)arate experimental estimates for v;omen, men, whites, 
blacks, Hispanics, wel£&re recipients, high school dropouts, and so on. 

The experimental analysis for the study follows standard statistical practice and uses 
multiple regression to increase the statistical precision of the program impact estimates. 
Ordinary least squares regression is used for those impact estimates based on continuous 
outcome variables, such as earnings; and maximum likelihood logistic regression is used for 
those impact estimates based on dichotomous outcomes that arc naturally expressed in 
percentage terms, such as «;q)loyment rates. These multivariate techniques control for 
chance differences between the treatment group and control group in a wide range of baseline 
characteristics, wiiicfa are included in the regression model as covariates. Appendix D 
describes the procedures employed in each case. 

But not all treatment group members ultimately became enrolled in JTPA. The estimated 
impacts per JTPA assignee therefore do not measure the effect of actually participating in 
JTPA. Instead, they measure the average effect of making the program accessible to eligible 
applicants, and thus they represent the actual impact that local programs can have by making 
JTPA Title II-A services available to economically disadvantaged members of the commu- 
nity.*^ 



Impacts per JTPA Enrollee 

As just noted, the estimated JTPA impact per assignee represents the average effect of the 
program on all treatment group members, whether they became enrolled in JTPA or iK)t. This 
result can be expressed as a weighted average of the impact on those who were enrolled plus 
the impact on those w4io were not enrolled, wiiere the weights are the proportion who were 
eiu'olled and the proportion who were iK>t. 



14. Note, however, that because coDtrol group members could and did obtain employment and training 
services from non-JTPA providers, the comparisoD of outcomes for treatment group and control group members 
represents the incrtmental effect of JTPA services relative to the services that could have been received 
elsewhere in the area. 
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If the program had no efifect on those who did not become enrolled, the impact per assignee 
understates the impact per cnroUec because the impact per assignee is the average of a zero 
impact per nonenroUee plus the average impact per cnrollec. In this case, to infer the average 
impact per JTPA enrollee, one need only divide the impact per assignee by the proportion of 
assignees who were enrolled (sec BlocHn, 1984a)." 

For example, if the average impact per assignee were $500, and 60 percent of the 
treatment group were enrolled in JTPA after random assignment, the estimated impact per 
JTPA enroUec would be $500/.6, or $833. TTius, estimated impacts per enroUec are 
proportional to estimated impacts per assignee. Inthis example, the 60 percent enrollment rate 
implies an estimated impact per enroUec that is 1/.6 or 1.67 times the estimated impact per 
assignee. 

To the extent that treatment group members who did not become enrolled in JTPA were 
not afifectcd by the p. '^Hgram, one can interpret our estimates of the impact per JTPA cnrollee 
as the average efifect of enrolling in a JTPA Title II-A program relative to what the cnroUees 
could have accomplished if they had not enrolled in the program. 

Nevertheless, to the extent that noncnrolled treatment group members experienced 
program impacts similar to those of cnrollecs, our estimates will systematically overstate the 
true impacts per cnrollec.'* Indeed, when interpreting the estimated in^acts per enroUec, the 
reader should note that some members of the treatment group vibo were not enroUcd ia JTPA 
did receive limited JTPA servi'^ js. This occurrence reflects a practice by same SDAs of not 
enroUing applicants immediately after they are judged eligible for the program. 

To investigate the extent to which these treatment group nonenrollecs received JTPA 
services, we conducted a separate analysis based on checks of SDA administrative records for 
a small subsample of treatment group members and on discussions with SDA staff members 
about \s*iat happened to nonenroUccs in this subsample (see ^pendix F). The administrative 
records indicated that about 40 percent of the subsample were not later cnroUed in 
Title n-A. Discussions with SDA staff about those nonenrollecs indicated that about half 
received no JTPA service and half received some service. The specific program services 
received were usually limited, b'^w >ver, mairdy omstituting attempts to arrange services for 
applicants by referring them to potential employers for (m4he*job training; by providing some 
job search assistance; or by attempting to arrange classroom training. Thus, with few 



15. An additiond adjustnicnt w«s made f(r the fiKt thit 3 percent 

despite the cxpcrimcnf$ cmbirgo on their ptfticipation. This •djustrocnt, however, bad almost no effect on the 
resulting cslimtlcs. For simplicity, then, wc ignore the adjustment in the following discussion. 

1 6 . In the unlikely event that JTPA enroUees and nooenrollees in the treatment group experienced opposite 
impacts from the program, our estimates of program impacts per JTPA cnrollec would understate the true 
impacts pci cnrollee. 
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exceptions, it is unlikely that the services treatment group noncnrollccs received appreciably 
affected their future labor market experience, although we cannot be sure about the e£fect. 
(See Appendix F and, for more detail, Doolittle, forthcoming.) 

We therefore consider our inferred estimates of impacts per JTPA enrollee as likely upper 
bounds on the magnitude of the true impact of enrolling in the program. And at the opposite 
extreme we consider our estimates of impacts per JTPA assignee as likely lower boimds on 
the magnitude of program impacts on enroUees . In the earlier example, then, the lower bound 
on the true impact on enrollees would be SS09 (the impact per assignee) and the upper bound 
would be $833 ($500 divided by the 0.6 enrollment rate). 



The Incremental Impact of JTPA 

Whether the impacts are reported per JTPA assignee or per JTPA enrollee, they reflect the 
incremental effect of JTPA services beyond v/bat sample members could have accomplished 
without access to JTPA, that is, with access to services from non-JTPA providers. In other 
words, they reflect the effect of adding JTPA semce^ to the existing landscape of employment 
and training programs in the community or region. 

The effect of those non-JTPA services is reflected in the labor maricet outcomes of the 
control group members. Hence, our impact estimates do not reflect \vbaX would h^pen in the 
absence of any employment and training services, but rather v/hat would happen without 
JTPA services. The difference between the outcomes of the treatment group and the control 
group therefore reflects the effeaof the increment in services JTPAavailed to treatment group 
members. 

To measure this increment in services, we measured the employment and training services 
received by treatment and control group members from JTPA and from other providers. In 
chapters 4 through 7 we report the difference in service receipt between the treatment group 
and the control group to illustrate the size of the increment that produced the program impacts 
estimated. Our forthcoming report will also include a detailed r * 'sis of the costs of these 
services and compare tlie added costs of additional services proviv ^ ^ to the estimates 
of program impacts. This will form the basis for a benefit-cost . Jie program. 



Impacts on Attainment of a High School Credential 

One key program outcome, especially for youths, is the attainment of a high school diploma 
or General Educational Development (GED) certificate. Using responses to the First Follow- 
up Survey, we identified sample members wiio both participated in a school or training 
program and auained a high school credential at some time during the ISnnonth follow-up 
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period. By comparing the percentage of treatment group members who received such a service 
and attained this credential and the corresponding percentage of control group members, we 
estimated the impact of JTPA Title II-A programs on this outcome. 

We refer to this finding as the program impact on attainment of a training-related high 
school credential Estimates are presented for each target group, both as a percentage of all 
assignees and as a percentage of assignees who were high school dropouts v^en they applied 
to JTPA. Impacts as a percentage of the larger group indicate the extent to which JTPA 
increased the educational attainment of the target group as a whole; impacts as a percentage 
of high school dropouts isolat : uae impact on only Aose target group members for whom such 
an effect was possible. 

Impacts on Earnings, Employment, and Earnings Components 

Beyond educational attainment, we also present JTPA impacts on sample members' earnings, 
their employment, and the components of their earnings. Estimates of impacts on earnings 
serve as the most comprehensive measure of the ability of the program to increase the labor 
market success of low^income, economically disadvantaged persons. Estimated impacts on 
employment measure the program's effect on disadvantaged persons' ability to find and hold 
jobs. And estimated impacts on the components of earnings indicate the extent to which 
program-induced earnings gains were the result of an increase in the amount of time woriced 
or an increase in the amount paid for time worked. The analysis of earnings components thus 
helps to explain how earnings gains (or losses) were produced for those groups that 
experienced a program impact. 

For each target group and service strategy subgroup within each target group, we report 
estimated impacts on earnings separately for each quarter after random assignment and in 
total for the 18-month follow-up period. The earnings impacts arc expressed in dollars and 
as a percentage of the average earnings of the control group. 

Impacts on employment, again for each quarter and the follow-up period as a whole, are 
reported in terms of three different but related measures of employment: 

♦ the percentage of the group employed at any time during the period, 

♦ the average number of weeks worked by members of the group during the period, 
including zeros for persons who did not work, and 

♦ the average number of hours worked during the period, including zeros for 
persons who did not work. 
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Finally, the average earnings of sample members are expressed as the product of four 
components: 

• the percentage of the group employed at some time during the period (termed 
workers per assignee), 

• the average number of weeks woriced by persons who were employed (termed 
weeks worked per worker), 

• the average number of hours worked per week worked, and 

• the average earnings per hour worked. 

The relative magnitudes of program impacts an these components were estimated as described 
in Appendix D and are preswited in ch^tcrs 4 through 6 to describe how earnings impacts 
were produced. These estimates also reveal the extent to vMch JTPA increased sample 
members' ability to find and hold jobs, the mix of part-time and full-time employment they 
obtained, aiid the wage rates they were paid. 

Data for the Report 

The data used to produce the impact estimates in this report come fi-om five main sources: 

• a Background Information Form completed by sample members (with assis- 
tance firom local SDA staff members if necessary) when they applied to JTPA; 

• First Follow-up Survey interviews that asked sample members about their 
earnings, employment, and receipt of employment and training services; 

• enrollment and tracking data from the 16SDAs, which include information on 
enrollment, service receipt, and termination status; 

• state unemployment insurance records on the quarterly wages paid to sample 
members by local employers; and 

• the national Job Training Quarterly Survey (JTQS) of JTPA participants' 
backgrounds and program experiences, drawn fi-om the administrative records of 
a nationally representative sample of SDAs. 
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The Background Infonnation Form is the primary source of baseline information on 
sample members. Administered as part of the program application process at each site over 
the course of the sample intake period, the form obtained data on applicants' demographic 
characteristics, education and training, employment history, Uving situation, and public 
assistance experience, as well as contact infonnation for the follow-up interviews. 

Data from the Background Information Form were used in this report for three mam 
purposes: to describe the 18-month study sample: to define the sample subgroups for which 
separate impact estimates were calculated; and to construa variables to control for treatment- 
control group differences in the multiple r^rcssion models. 

The First FoUow-up Survey, as noted earlier, was scheduled for the full experimental 
sample for periods that varied across sample members from 13 to 22 months after random 
assignment. This report is based on the experiences of those sample members scheduled for 
at least 18 months of foUow-up (the 18-month study sample), who made up over four-fifths 
of the fiill experimental sample." 

As described in more detail in Appendix C, the First FoUow-up Survey was a 30-minute 
interview that asked sample members about their earnings, employment, and receipt of 
employment and training services from the time respondents were randomly assigned to the 
date of their interview. It also asked questions about current femily composition and related 
issues. The survey was conducted by telephone, with in-i>erson interviews for sample 
members who could not be reached by telephone. 

Exhibit2.5 shows that the response rate for the First Follow-up Survey was 84.8 percent, 
which is unusually high, especially for low-income persons. Response rates for females 
approached 90 percent and were higher than those for males. But even the lowest response 
rate in the exhibit, 75.7 percent for adult men recommended for the other services strategy, 
was higher than the rates in many previous studies of economically disadvantaged Americans. 

First Follow-up Survey dato provided the outcome measures used to estimate program 
impacts on earnings, emplo>ment, and the components of earnings . Appendix D examines the 
extent to which survey nonresponse biased these impact estimates, using earnings data from 
stote unemployment insurance (UI) agencies for both survey respondents and nonrcspondents. 
This analysis indicates that survey nonrcspOTse bias did not affect the estimates of canungs 
and employment impacts for adult men and out-of-school youths. For adult women, however, 
survey nonresponse introduced a small bias, which we adjusted for in the impact estimates 
using UI earnings data for survey nonrcspondents (see Appendix D). 



17. The Second Follow-up Survey has been •dmimstered to about a third of the experimental sample. 
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Exhibit 2.5 Response Rates in the First Follow-up Surrey: IS-Month Study 
Sample, by Target Group and Sendee Strategy Subgroup 







Percentage 


responding to 


the survey 






Full 


Adult 


Adult 


Female 


Male ^ 


Service strategy 


sample 


women 


men 


youths 


youths 


subgroup 


(1) 


(2) 


(3) 


(4) 


(5) 


Classroom training 


86.1% 


88.1% 


80.6% 


88.4% 


83.87c 


OJT/JSA 


86.2 


89.3 


82.7 


90.0 


86.2 


Other services 


81.2 


86.4 


75.7 


85.1 


78.7 


All subgroups 


84.8 


88.2 


80.3 


S7.7 


82.7 



Sounds: Unadjusted frequencies based on Backgrxjund Information Form responses and First 
Follow-up Survey responses. 
a. Out-of-»chool youths only. 



Computerized administrative records from the 16 SDAs in the study provided information 
on JTPA enrollment rates for sample members, the amount of time JTPA enrollees spent in 
the program, the specific program services they received, and whether they were employed 
when they left the program. 

These enrollment and tracking data were used to describe the JTPA program service;, 
received by sample members, to compare the specific program services received by persons 
recommended for the three service strategies, and to identify control group members who 
entered JTPA during their 18-month embargo period (3 percent did so). 

We used quarterly earnings data from state UI agencies for 1 4 of the 1 6 sites in three ways 
(see Appendix D).^* First, we used UI data for the five quarters before random assignment 
to construct variables to control for treatment-control group differences in pre-random 
assignment earnings in the multiple regression models used to estimate program impacts. 
Second, as mentioned above, we used UI data for the first four quarters after random 
assignment to ascertain any survey nonresponsc bias m the survey-based estimates of program 
impacts on earnings. And finally, we used UI data for the first six quarters after random 
assignment (the full 1 8-month follow-up period) to adjust for nonresponse bias in the survey- 
based estimates of impacts on the earnings of adult women (the only target group, as noted 
above, for which nonresponse bias was large enough to warrant the adjustment). 



1 8. We were unable to obtain stale UI records for the Marion, Ohio, site. In addition, problems with tlic 
UI records we received for the Jersey City, New Jersey site precluded using thai information in this report 
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Our last data source, the Job Training Quarterly Survey (funded by the U.S. Department 
of Labor, conducted by the U.S. Bureau of the Census, and reported by Westat, Inc.), collects 
background information on JTPA enroUees and terminees from a standing sample of 142 
SDAs, selected to provide a statistically valid representation of the JTPA Title II-A and Title 
m programs nationally. Drawn quarterly from on-site file searches at each participating 
SDA, JTQS data are the primary source of current information about the number, character- 
istics, and program services of JTPA enrollees and terminees around the country. We used 
these data to compare our 18-month study sample and the population of adults ;uid out-of- 
school youths in JTPA Title II-A nationally during the sample intake period for this study. 
This analysis, reported in Appendix B, compares the two groups on the basis of JTQS data 
on their baseline characteristics, the duration of their enrollments, the mix of program 
services they received, and their labor market status upon termination from the program. 
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Context: Study Sites, 18-Month Study Sample, and 
Patterns of JTPA Enrollment and Service Receipt 



THIS chapter describes the study sites and sample for this report. The first section 
catalogs characteristics of the 16 service deUvery areas (SDAs)^ that agreed to 
participate in the national JTPA study and compares those characteristics — of the local 
population and economy, the JTPA programs in place, program participants, and program 
services — with averages for the broader group of 649 SD As nationally during the years the 
study sample was selected. 

The second section of the chapter presents a more detailed examination of the 1 7,026 
members of the 1 8-month study sample and the composition of the main subgroups analyzed 
in this report. Finally. . last section details pattems of JTPA enrollment and service receipt 
among these subgroups. All of these topics are analyzed in more depth m the companion 
volume on the study' s implemf^ntalion (Doolittle, forthcoming). 



The 16 Study Sites 

As noted in Chapter 2, the 16 study sites were recruited from among those SDAs in the 
continental United States with at least 500 persons ending their enrollment in Title II- A- 
funded services ('lerminees" in the language of program reporting ) in program year 1984.^ 



1. In JTPA parlance "service delivery area" refers to both the local administrative agency for the 
program and the geographical area it serves. Most SDAs provide some sptcific program service themselves, 
but many also contract with other providers of employment and training services. 

2. Program year 1984 (July 1984 through June 1985) was the most recent year for which data were 
available at the time site selection for the study began. 
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Exhibit 5. 1 Key Facts about the 16 Study Sites 



Size 
of the 
IB-monih 



Site name 


SDAname 


Census 
region 


Largest 
city 


study 


Fort Wayne, Ind. 


Northeast Indiana 


Midwest 


Fort Wayne 


2,559 


CooM Valley, Ga. 


Coosa Valley, Ga. 


South 


Rome 


1,806 


Corpus Chr'jd, Tex. 


Coipur Christi/Nueces 


South 


Corpus Christi 


1,498 




County, Tex. 






I ♦375 


Jackson, Mist. 


Capital Area, Miss. 


South 


Jackson 


Providence, R.I. 


Providence/Cranston, R.I. 


Northeast 


Providence 


1 

IfZ/ f 


Springfield, Mo. 


Job Council of the 


Midwest 


Springfield 


l,2uZ 




Ozaiks, Mo. 








Jersey City, N.J. 


Jersey City, N.J. 


Northeast 


Jersey City 


1, 1 /u 


Marion, Ohio 


Crawford/Hancock/ 


Midwest 


Marion 


1,083 




Marion/Wyandot Counties, 










KJtuO 








Oakland, Calif. 


Oakland, Calif. 


West 


Oakland 


1,043 


Omaha, Neb. 


Greater Omaha, Neb. 


Midwest 


Omaha 


956 


Laf imcr County, Colo. 


Larimer County, Colo. 


West 


Fort Collins 


668 


F ^land, Fla. 


Heartland, FU. 


South 


Lakeland 


597 


Northwest Minnesota 


Northwest MinnesoU 


Midwest 


Thief River Falls 


498 




(Crookston and Thief River 










Falls) 








Butte, Mont. 


Concentiated Employment 


West 


Butte 


477 




Program, Mont. 








Decatur, 111. 


Macon/De Witt Counties, 111. 


Midwest 


Decatur 


471 


Cedar Rs^ids, Iowa 


East Central Iowa 


Midwest 


Cedar Rspids 


346 


AU sites 








17,026 



The map in Chapter 2 (Exhibit 2. 1) shows the location of these sites and lists the abbreviated 
site names xised in this report The formal lumie, census region, and largest city of each SD A, 
and the size of each SDA, and the size of the 1 8-month study sample from each, are shown 
in Exhibit 3.1. Throughout this chapter, the exhibits list the sites in descending order by the 
size of the study sample at the site. 

These 1 6 sites are spread throughout the nation, with two in the Northeast, four in the 
South, seven in the Midwest, and three in the West They include sites located in large 
metropolitan areas with large minority populations (Jersey City and Oakland), others located 
inpredominately rural areas or small towns (Coosa Valley, Georgia; Marion, Ohio;Northwest 
Minnesota; Butte, Montana), and still others with a mixture of urban, suburban, and rural 
areas (Fort Wayne; Omaha; Decatur, Illinois). The size of the largest city within each of 
these SDAs ranges from 372,000 in Oakland and 336,000 in Omaha to under 10,000 in 
Northwest Minnesota's Thief River Falls (1990 U.S. census data). 
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No large central cities are included among the study sites. JTPA operations m many 
central cities aredecentralized,%vith service providersplayinganimportantrolemmtakeand 

assessment. In Los Angeles,forexample,atthetimeofsiteselectionover50organizations 

were involved in client intake for the program. Because the research design involved 
coordinating the random assignmem with cliem intake and arsessment, and ma site like Los 

Angeles those procedures often varied slightly from office to office, the research team was 
uniletodevelop workablestudvproceduresforthistypeof site. Nevertheless, twosmaller 

SDAsinlargemetropolitanareas(JerseyCityandOakland)didparricipatemihestudy,and 
they have many of the same characteristics (in ter.ns of clients, economic conditions, and 
ser^'ice availability) as large central cities. 



Population Characteristics 

As shoxsn in Exhibit 3 .2, the sites were also quite diverse in population densities and povem- 
rates ' Three entirely urban SDAs stand out in population density (Providence, JerseN- Cit>-, 
and Oakland), while Northwest Minnesota and Butte, Montana, fall at the other ex^treme. 
Fort WaNTie's relatively low population density is an average of the c»ty °f Fort Wayne 
and the surrounding eight predominately rural counties that are also part of this SDA. The 
average for the 1 6 sites is above that for the nation as a whole, at least m part because rural 
SDAs with only a small number of participants were not recruited to participate in the stud>'. 

The povem- rates, shown in the right-hand column of the exhibit, show similar variet> / 
The sites contai^iing large metropolitan areas with largeminority populations— Jerse>'Cit>-and 
Oakland-had the highest poverty rates, but other sites with minonty populations such as 
Corpus Christi (largely Hispanic), Jackson, Mississippi (black), and Providence (black and 
Hispanic) also had higher-than-average rates. Two predominately rural sites (Coosa ValleN' 
Georgiaand Northwest Minnesota) and one with a mix of urban and rural (Heartland, Florida) 
had poverty rates slightly above the 16.site and national averages. 



Economic Conditions 

Economic conditions at the sites, summarized in Exhibit 3.3, reflect differences in both 
regional economic conditions and the local economic base. As is the case nationally, the 
averageunemploymemrates(columnl)masklargerdifrerencesamongthesites.» Corpus 

3. Mote that in this and the following exhibits, the present site and national averages are unweighted. 

4. The poverty rates reported in the JTPA Annual Status Report file are based on information from 
the 1980 census, which collected data on annual income in 1979. 

5. The uncmplov-ment rates presented are for the labor force living in the geographic area included 
m each SDA. 
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Exhibit 3.2 Selected Population Characteristics of the 16 
Study Sites 



Site 


square mile? 
1989 


Percentage of 
residents in 
poverty, 1979 


POlV •jrDC, AUVI* 


160 


5.9% 


woosft vftuey, 


110 


10.7 


corpus v^iinsu, xca. 


360 


13.4 


JftCKSOu, JMISS. 


360 


12.8 




4,680 


12.1 


SDrini?ficld. Mo. 


80 


10.1 


Jersey City, N J. 


7,000 


18.9 


Marion, Ohio 


120 


7.2 


Oakland, Calif. 


6,620 


16.0 


Omaha, Neb. 


550 


6.7 


Larimer County, Colo. 


70 


5.9 


Heaitland, Fla. 


100 


11.3 


Northwest Minnesota 


10 


11.1 


Butte, Mont 


10 


IS 


Decatur, 111. 


150 


7.8 


Cedar Rapids, Iowa 


90 


6.0 


l&site average 


1^79 


10.2 


National average, all SDAs 


733 


9.7 



Soufce: Unweighted uKual aveimget calculated fiom JTPA Annual Sutut 
Report compuUr filet produced by the U5 . Department of Labor. 
. Of the tenrice delivery area (SDA). 



Christi*sresideiits,for example, experienced persisteiitly high unemplo^ 
1 980s, as the oil industry suffered an extended slump. At the other extreme, Providence's 
low unemployment rate was the result of fte New England region's high technology boom 
of the same period, while the low rates in Fort Wayne, Omaha, and Cedar Rapids reflect 
the economic resurgence some mid-sized metropolitan areas in the Midwest wer. enjoying 
at the time. Decatur's high unemployment, however, illustrates that the recoveiy was not 
ubiquitous; in this manufacturing andfoodproccssingcenter, the recovery of thenwd 

1980s was weak. 

The variation in the average earnings of the population in each site (column 2) reflects 
in part the wage disparities between urban areas (for example, Oakland versus Northwest 
Minnesota) and in part the concentration of high wage industries in some sites (petroleum 
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Exhibit 3. 3 Selected Economic Conditions at the 16 Study Sites 



Site 



Mean 
unemployment 
rate, 1987-89 
(1) 



Mean 
earnings, 
1987 
(2) 



% employed in 
manufacturing, 

mining, or 
agriculture, 1988 
(3) 



Annual growth 
in retail and 
wholesale 
earnings, 1989 



Fort Wayne, Ind. 
Coosa Valley, Ga. 
Corpus Chrifti, Tex. 
Jackson, Miss. 
Providence, R.I. 
Springfield, Mo. 
Jersey City, N.J. 
Marion, Ohio 
Oakland, Calif. 
Omaha, Neb. 
Larimer County, Colo. 
Heartland, Fla. 
Northwest Minnesota 
Butte, Mont. 
Decatur, 111. 
Cedar Rapids, Iowa 

l&site average 
National average, 
allSDAs 



4.7% 
6.5 
10.2 
6.1 
3.8 
5.5 
7.3 
7.0 
6.S 
4.3 
6.5 
8.5 
8.0 
6.8 
9.2 
3.6 

6.6 
6.6 



S 18.700 
16,000 
18,700 
17,600 
17,900 
15,800 
21,400 
18,v500 
23.000 
18,400 
17,800 
15.700 
14,100 
16.900 
21,100 
17.900 

18,100 
18,167 



33.3% 

42.8 

16.8 

12.8 

28.0 

19.4 

20.9 

37.7 

14.6 

11.8 

21.2 

23.8 

23.0 

9.6 

27.1 
21.9 

22.8 
23.4 



^.1% 

2.1 
.15.5 

-2.4 

9.7 

-1.8 
9.9 
1.7 
3.0 
1.8 

-3.1 

.0.3 
2.4 

-5.7 

.1.1 

.0.5 

0.0 



1.5 



S^Htfce: Unweighted aniBud avengec calculs^ed from JTPA Annul Sutus Report computer filet proAicod 
by Uie U.S. Dcptitmetft of Labor. 

Note: Miuii^s dau for certain meMurea precluded uaiog the aaine year acn>«s columns. 



in Corpus Christi and heavy manufacturing in Fort Wayne and Decatur).^ These dififerences 
in the local economic base are furfliur iUus'jrated in column 3 of Exhibit 3.3, which displays 
the percentage of workers employed in the goods-producing industries of manufacturing, 
mining, and agriculture. 

The last column of Exhibit 3 . 3 , on annual growth in retail and wholesale earnings during 
1 989, C2q)tures the effects of economic conditions in that year on each SD A. Corpus Christies 
economic downturn is starkly visible ( a negative 15.5 percent), as is the economic 
boomintheNortheastduringthelatel980s(secJerseyCityv^dthagrowthrateof9^ 
andProvidencewith9.7percent). Nevertheless, on all of tfiese measures the 16-siteaverage 
is quite similar to the natiorud average for all SD As. 



6. Avenge earnings lie ctlcul4iod by dividing the toul payroll reported by cn^loycis in the SDA to 
fcdcTBl and sUte unemployment insurance agencies by the number of employees in the SDA. 
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Exhibit 3.4 Selected Characteristics ofTitU lUA Terminees at the 16 Study Sites, 
Frogram Years 1987-19S9 







Perceniaze of all 


terminees 






Youths, 


mue. 


Black. 






ages 


non- 


non- 


Hispanic 




14^21'' 


Hispanic 


Hispanic 


Site 


0) 


(2) 


(S) 


\4) 


Fort Wayne, Ind. 


A6% 


74% . 


22% 


3% 


Coosa Valley, Ga. 


A3 




20 


0 


Corpus Christi, Tex. 


45 


21 


8 


71 


Jackson, Miss. 


58 


14 


85 


0 


Providence, R.L 


45 


34 


38 


21 


Spnngfield, Mo. 


39 


95 


3 


1 


Jcricy City, N.L 


55 


5 


68 


Marion, Ohio 


41 


95 


3 


2 


Oakland, Calif. 


44 


7 


68 


6 


Omaha, Neb. 


37 


42 


51 


4 

17 


Larimer County, Colo. 


20 


78 


2 


Heartland, Fla. 


42 


57 


37 


5 


Northwest Minnesota 


47 


95 


0 


3 


Butte, Mont. 


39 


90 


0 


3 


Decatur, 111. 


44 


eo 


39 


0 


Cedar Rapids, Iowa 


50 


87 


9 


1 


JS-site average 


44 


58 


28 


10 




44 


61 


26 


10 



Source; Uawci^ud taiuai «v€nf « calcuUiisd from JTPA AbwuI Sucui Re^wrt oompuiCT filet pxalucod 
bythcU.S. Dcp«tmentpfLibor. . , . • 

«. ladudci bo<h ogi-cf-t<*ool •ad in-tchoo! youhi. Tkc It-OKnah iiady SMople doc* i^udc tft-Kliool 
youths or youths uader sf e 16. 



Participant Characteristics 

Thesites also exhibited diversity in their program participants. Exhibits 3.4 and 3 .5 display 
selected characteristics of program terminees during the sample intake period.'' The large 
differences in ethnic distribution across the sites reflect differences both in the local 
population generally and in the populations eligible for JTPA. In particular, the SDAs with 
largemetropolitanareashaveamuchhigherproportionofblackandHispanictermine^ 
do SDAs in rural areas. The highest minority percentages are in Corpus Christi (71 percent 



7 TTie frequencies shown in Exhibits 3.4 and 3.5 are for ail JTPA Title H-A terminees during the 
program ycais listed and arc based on daU in the JTPA Annual Sutus Reports (JASR) compiled by the 
Department of Labor. These JASR daU are the best source of infonnaUon on individual SDAs and the 
people they serve, but they do not allow for separate breakdowns of out-of-scbool and in-school youths. 
The latter group was excluded from the National JTPA Study (as explained in Chapter 2). Appendix 
B presents an alternative analysis comparing JTPA cnrollees and tenninecs nationally, based on data from 
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ExhibiJ 3 J Selected Barriers to Emplcym^nt Faced by Title II-A Terminees ax the 16 Study Sites, 
Program Years 1987-1989 









Percentage of all terminees ^ 








Long-term 


High 


Unemployed 




Physical 


IWiJUmx Cm 






AFDC 


school 


IS-^ weeks 


Limited 


or menial 


<7ih grade 


Ever 




recipients^ 


dropouts^ 


in past 26 


tJigUsn 


aisoDiuiy 


level 


arresied 


Site 


(1) 


(2) 




(4) 

IT/ 


(5) 


(6) 


0) 




2% 


25% 


33% 


0% 


15% 


17% 




Cnosa Vallcv. Ga. 


4 


42 


8 


0 


14 


31 


2 


Corpus Chtisti, Tex. 


8 


41 


4y 


1 


A 
•t 


33 


8 


Jackson, Miss. 


21 


25 




A 

u 


1 1 
1 1 


21 


3 


Providence, R J. 


7 


37 






A 
H 


29 


7 


Springfield, Mo. 


3 


28 


34 


1 


3 


7 


4 


Jersey City, NJ. 


7 


11 


43 


3 


4 


4 


3 


Marion, Ohio 


9 


25 


73 


0 


23 


23 


7 


Oakland, Calif. 


25 


17 


25 


17 


11 


25 


5 


Omaha, N^. 


16 


18 


49 


0 


10 


9 


4 


Larimer County, Colo. 


3 


21 


57 


1 


14 


12 


5 


Heartland, Fla. 


6 


30 


7 


1 


13 


17 


13 


Northwest Minnesota 


13 


12 


41 


0 


17 


10 


4 


Butte, Mont. 


1 


15 


64 


0 


19 


5 


10 


Decatur, lU. 


16 


11 


39 


0 


10 


26 


10 


Cedar Rapids, Iowa 


11 


18 


74 


2 


23 


11 


6 


l&site average 


10 


25 


43 


3 


12 


18 


6 


National average, all SDAs 


9 


25 


40 


3 


14 


21 


8 



Source: Uawei^hud amual tvcnget calculated from n?A Annual Sutua Report computer fUea produced by the U5 . 
Department of Labor. 

a. Includea adulu and both out-of*achool and in-achool youths agea 14 to 21 . The IS-OMOth atudy aample doea not include 

in-acbool youtha oryouthi luiderage 16. 
h. Family recesviqg AFDC for any 24 or more of the 30 oKxttha preceding determination of elipbility for HPA. 
c. No high achool diploma or General Educational DevelopoMnt (GED) certificate. 

Hispanic and 8 percent black), Jackson (85 percent black), Jersey City (68 percent black and 
21 percent Hispanic), and Oakland (68 percent black, 6 percent Hispanic, and 19 percent 
other minorities, mostly Asian not shown). On average, though, the etlmic composition of the 
sites practically mirrored that of SDAs nationally. 

Exhibit 3.5 shows the proportion of all terminees who were facing one of seven selected 
barriers to employment. Again, the averages for the 16 study sites are very close to the 
averages for all SDAs. Furthermore, in both the study sites as a group and all SDA 

the Job Tniuing Qvuulcrly Survey. The appendix indicates that the members of the 1 S-month study sample 
were somewhat less likely to have a high school diploma or GED certificate when they applied to JTPA 
than were their national counterparts. 
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nationally, the proportion of terminees facing any one of these barriers was relatively low. 
The only exception was the barrier of limited recent work experience (column 3), which 
affected on average 43 percent of terminees at the sites and 40 percent nationally. 

The si*e rankings on these employment barriers varied substantially across the measures. 
For example, terminees in Coosa Valley, Corpus Christi, and Providence had the highest 
incidence of educational barriers (columns 2 and 6), while terminees in Marion, Butte, and 
Cedar Rapids were the most likely to have limited recent work experience and aphysical or 
mental disability (columns 3 and 5). 



Exhibit 3.6 Selected Characteristics of JTPA Tule H-A Programs at the 16 Study Sites. 
Program Years 1987-1989 



Site 



Mean number 
ofadulxand 
youth terminees 

(1) 


Mean number 
of weeks enrolled 
Adults Youths" 

a) (3) 


Mean federal 
program cost per 
adult terminee 


1.195 


16 


31 


$ 1.561 


1.063 


12 


15 


2,481 


1,049 


34 


33 


2,570 


1.227 


8 


15 


1.897 


503 


7 


5 


2.841 


938 


17 


17 


1.898 


853 


16 


14 


3.637 


714 


27 


26 


2.199 


1.396 


16 


17 


2,539 


1,111 


11 


12 


2,404 


354 


32 


26 


1.937 


1,793 


IS 


24 


1.782 


430 


29 


28 


2,371 


576 


21 


19 


2.665 


525 


29 


25 


3,039 


658 


31 


23 


2.212 


899 


20 


21 


2,377 


1,177 


20 


22 


2,241 



Fort Wayne, Ind. 
Coosa Valley, Ga. 
Corpus Christi, Tex. 
Jackson, Miss. 
Providence, K.I. 
Springfieki, Mo. 
Jersey City. N.J. 
Marion, Ohio 
Oakland, Calif. 
Omaha, Neb. 
Larimer County, Colo. 
Heartland, Fla. 
Northwest Minnesota 
Butte, Monl. 
Decatur, III. 
Cedar Rapids* Iowa 
IS-site average 
National average, all SDAs 



Source: Uovvvif htod timual tver^et cdcuUtod from JTPA Annutl Sutua Report 
the U.S. Department of Labor. 

A. IncJudea adulu md both out^f^hool and iD-achool youtha agea 14 to 21. The 
include in-KbooI youtha or youtha under 9gt 16. 



oomputer iilea produced by 
Ift-month atudy aample doei not 
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Program Characteristics 

Title n-A operations differed widely across the sites in terms of the size of the program, the 
average duration of program services, and program costs. As shown in Exhibit 3.6, the 
programs ranged in size from 3 54 terminees annxially in Larimer Coiinty, Colorado to 1 ,793 
in Heartland, Florida, over the two-year period.' The range for tiie average length of time 
terminees spent in the program was also large: for adults the average number of weeks 
enrolled ranged from a low 7 in Providence to a high of 34 in Corpus Christi, while the 
range foryouths was from 5 to 33, in the same two cities. Average annual federal costs per 
adult terminee reflected both the differences in the lengths of enrollment and the hi^er 
service costs (based onhigher oflSce rental and salaiy rates) in large metropolitan areas such 
as Jersey C ity and Oakland . Note that the sites as a group differed from the national average 
in program size (number of terminees), again because the study did not include very large 
SDAs. 

SDAs have the responsibility for selecting and defining the roles of other organizations 
that will provide JTPA-fimded services. These providers range from other ^ blic agencies 
to community-based and other nonprofit organizations to proprietaiy schools and private 
sector firms. Exhibit 3.7 displays the variety in service providers that contracted with the 
1 6 SDAs to supply employment and training services during the sample intake period. 

Publiceducational institutions — vocational-technical schools, community colleges, and 
universities — provided classroom training in 1 4 sites, and proprietaiy schools were providers 
inhalf of the 16 sites. Arrangingfor subsidized on-the-job training(OJT) positions in the 
private sector was done by SDA staff memb ers themselves (8 SDAs) or with the assistance 
of the state job service, which played arole in 2 of the SDAs; in aiuurth acommunity-based 
organization was also involved. The j ob service alone arranged for OJT in another 2 SDAs, 
and acommunity college arranged for the service in 1 SDA. In 2 SDAs private sector firms 
aiTi jiged for some OJT positions. A wide variety of organizations provided JTPA-fiinded 
job search assistance, although the SDA, the job service, or both were the most common.* 
Basic education was provided by public schools or community colleges in 9 of the 12 sites 
offering it. The remaining 4 SDAs did not offer basic education as a discrete, identifiable 
service. 



8. The tverage number of tenninecs annualiy during the period of the study's random assignment 
is not related to the sample size in each SDA in any simple way, because the duration of the random 
assignment varied across sites. 

9. Two SDAs (Providence and Cedar Rapids) did not offer job search assistance as a discrete service, 
instead ofTering it only as an integrated part of other services. 
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Exhibit 3,7 Most Common Service FroviJers Used by JTPA TitU H-A Programs at the 26 Study Sites, by 
Specific Program Service 



Site 



Specific progrwn service 



Classroom trcdning 
in oecupaticndl 
shUs 
(1) 



trmning 

a) 



Job search 



(3) 



Basic 
tducasion 
(4) 

Noi provided 



Foct Wayne, Ind. 
CooMVaUcy, Ga. 
Cofpui ChriiU, Tex. 

Jackjoo. MIm. 

Provideoce, R.L 
Sprii^cld, Mo. 
Jeney City. N J. 

Marion, Ohio 
OaUaad, Calif. 

Omaha, Neb. 



Proprieuiy acbool 

Vocational* 
technical acbool 

Commmity coll^ 

Vocational* 
tachjucal acbool 

Cooununity-baaed 
origanizatioQ 

Community college 

Proprieuiy acbool 

Community-baaed 
or^ganizatioo 

Community college 

Proprieuty acbool 

Comtnunity-boed 
organization 

ProprieUfy acbool 

Vocational- 
technical fcbool 

Community*baaed 
orgaazitioo 

Proprieuty acbool 

Vocational- 
technical acbool 

Comnainity college 

Vocatiooal- 
technical acbool 

Comnmtnity-baaed 

organzatioo 
Proprieucy acbool 
SDA 

Community-baaed 
ocganizMion 
Community college 
Proprieucy acbool 



SDA 



Comounity-baaed 

ocgan&zattoo 
Private aectot firm 

Private aector firm 
Stale job aetvice 



Siaie job aervice 
SDA 



SDA 



SDA 



SDA 



SDA 



CofTdxainity-baaad 

organizatioo 
SDA 

SDA 



Comnxmity-baaed 

ocgaozatioo 
SDA 

Comininity-baaed 
ocganiztfioa 

Comnmmity-baaed 
ocganizatioo 



Comnamity^aaed 

ocganization 
Stale ufuvcraty 

Not provided 



SDA 



Commucity-baaed 

organization 
SDA 



SDA 



Commmity'baaad 

ocganization 
Propriauty acbool 
SDA 

Comimnity-baaed 
ocganizMion 
SDA 



Community college 
Community college 

Public acbool 

Not provided 



Public acbool 
Vocational- 
technical acbool 

Proprieuiy adtool 



SDA 

CoouBunity-baaed 
organization 

NotprtTvided 



(OmtinMd) 



Hi 
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Exhibit 3. 7 Most Commcn Service Providers Used by JTPA Title ll-A Programs ax the 16 Study Sites, by 
Specific Program Service (continued) 



Specific program service 





Qassroom traimng 










in occupational 


On-the-job 


Job seardi 


Basic 




skiiis 


traimng 


assistance 


education 


Site 


(1) 


(2) 




W 


Larimer County, Colo. 


Vocational 


SPA 


State job service 


Public school 




technical school/ 




SDA 


Vocstional 




convnunity cotlege* 






technical school/ 
community college 


HeaitlAnd, FU. 


Cotnnunity college 
Public School 
Proprietaxy school 
Vocational* 
technical school 


SDA 


State job seivice 


Commutnty college 
Public school 


Noithweflt Minne«oU 


Community college 
State univcnity 
Vocsticnal- 
teduiictl school 


Stale job 
service 


State job seivicc 


Not provided 


Butte, Moft. 


Comminty^bssed 

organization 
Comnvnity college 
Public school 


State job 
acfvioe 


State job aervicc 


Public school 


Decatur, 111. 


Comnunity college 


Community 
college 


Community college 


Public school 


Cedar Rjpida, lowt 


Convnimty college 
Vocational- 
technical school 


State job 

aervioe 

SDA 


Not provided 


Commuiuty college 



Source: Information collected by Manpower Demonstration Research Corporalion (Part A contractor of the 
National JTPA Study ), during SDA viaiu. 

Note: Information on the laat two oategoriea of program senicea-work e3q)cricnoe snd miaoellaneoua seryioei- 
examined in this report is not shown becauae the former waa rarely offered and the latter were too numerous 
to repreaent here. 

a. InLarinierCounty the vooational^echnical school becanne a community ooUegeduru 



A further important choice facing SDAs was uiiether to provide classroom training in 
occupational skills by referring clients to training providers; by "purchasing"' a class for 
several clients at once through a contract with a service provider, or by pursuing both 
approaches. On the one hand, if an adequate service provider network existed, individual 
referrals all owed SDA staffto match the training to the interests and needs of specific clients 
(rather than recruit to fill a class); referrals also avoided the possibility of hooding'' 
specialized labor markets with numerous program completer^ at once. On the other hand, 
purchasing a class allowed the SDA to exert more control over course offerings and 
curriculum and — if staff members were able to identify occupations in demand — ^to pinpoint 
JTPA training resources vsliere there would be a strong demand for graduates. 
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The 1 6 SD As in the study showed considerable diversit\' in how the\- made this strategic 
choice: 

• Six SDAs relied exclusively on individual referrals to training providers. 
These included three rural sites (Marion, Northwest Minnesota, and Butte) and 
three mixed urban-rural sites (Fort Wayne, Decatur, and Cedar Rapids), which 
relied primarily on public vocational-technical institutes or other public technical 
or community colleges. These sites tended to emphasize classroom training in 
occupational skills less than other sites. In virtually all target groups at each of 
these sites, the percentage of JTPA enrollees who received classroom training 
in occupational skills was less than the 16-site average for all JTPA enrollees in 
that target group. (The sole exception was adult men in Butte, Montana.)'^ 

• Four SDAs relied exclusively on contracts to purchase classes. The four 
(Jackson, Providence, Omaha, and Heanland) were in urban or mixed urban- 
rural areas and wrote from five to nine contracts each within a program year. 
The training was for occupations including truck driving, security guard, retail 
sales, automotive maintenance, food preparation, marketing, clerical, photocopy 
machine repair, and home health aide. Two SDAs v^th a much higher-than- 
average enrollment in classroom training in occupational skills (Omaha for all 
four target groups and Jackson for adults) fell into this category. 

• The remaining six SDAs used a mixture of individual referrals and class 
contracts. Three of these SDAs, which were in larger, urban areas (Corpus 
Christi, Jersey City, and Oakland) relied on communitj'-based organizations for 
training contracts in addition to public vocational-technical institutes, colleges, or 
proprietary schools. The other three, which included medium-sized towns and 
rural areas (Coosa Valley, Sprijigfield, and Larimer County) relied primarily on 
public vocational-technical schools and colleges. This category included some 
SDAs enrolling a larger-than-average percentage oftheir sample in classroom 
training in occupational skills: Jersey Cit\' for all target groups, Oakland for 
adults, and Corpus Christi for yoiilhs. 



10. Unfortunately, the enrollment and tracking data collected from most sites did not include 
information on the occupation for which people were trained, and the multiplicity of individual referrals 
prevented the research team from examining contracts for each training placement thai would identify the 
occupation. 

11. As for the first group, data limitations precluded a complete examination of the occupations 
involved. But the available information on class contracts at these sites suggests the trainmg was for 
occupations similar to those noted for the second group 
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Agency Standards and Program Performance 

As noted in Chapter 1, the Department of Labor has set certain standards for the 
performanceofJTPAservicedeUveiy areas. Exhibit3.8 lists the employment standard set 
foradults ineach of the 1 6 SDAs studied (tenned "predicted" performance) and their actual 
performance on that standard in program year 1988; Exhibit 3.9 does the same for two 
standards for youths: fte^'positive termination rate** and the "entered employment rate."" 
The columns labeled "difference** in die two exhibits indicate the difference between 
expected and actual performance on tliese three indicators." 

It is quite obvious that on all three measures the study sites include some that performed 
much better than the standard set for them, others that sli^tly exceeded theirs, still others 
that failed to meet theirs. On average, though, the 1 6 sites exceeded their predicted rate by 
an amount equal to or only slightly smaller than the amount SDAs nationally did. 



Services Received at the Study SrrES 

Although the study sites are similar tc SDAs nationally in many ways, they exhibit one 
important difference from their counterparts nationalty: thty emphasized classroom training 
and job search assistance more, and on-the-job trainLig and miscellaneous services less. 
AppendixBpresentsdetailed comparisons oftheservices received by nPAenroUces in t^^ 

1 8-month study sample and those received by JTPA tenninees nationally.^^ TTie pattem of 
more classroom training and job search assistance in the study sites than was the case 
nationaUy,andlessOJTandmisceUaneous services, was apparentforaUfourtarget^^^ 

The analysis of program impacts presented in this report partly controls for tiiese 
differences between services received by the study sample and those received nationally by 
JTPA participants, bypresentingseparate impact findingsforsamplesubgrou^^ 



12. The predicted pcrfonntncc levels are set by DOL regression models that control for the character- 
istics of both the SDA's labor market and its TiUe II-A tcxminecs. In roost, but not aU, of the sites 
these adjusted standards were the level against which states assessed local performance for the purpose 
ofallocalinginccntivegianU. In some cases the state agency woddmakefu^ 

produced by the regression model, to rencct ^ial circumstances not taken into account by that model. 

1 3 . The Sutc of Georgia chose not to uae the youth positive termination rate as a standard in program 
year 1988, and so the standard is not reported for Cooaa Valley. Similariy, Jersey CiQr, Omaha, and 
Larimer County were in sutes not using the youth entered en^>loyrocnt rate. In calculating the 16.sitc 
and national averages in these exhibit* we excluded any sites not using the standard in questton. 

14. App«. !ix B compares enrollment and tracking daU from 16 SDAs on the services received by 
treatment group roemben who were enrolled in JTPA during the foUowmppcriod and JobTraining Quarterly 
Survey data on the services received by JTPAtcrminecsnatiooally who were cnroUed in the program during 
the sample intake period for this study. 
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Exhibit 3. 8 Agency Performance Standards and JTPA Title U-A Performance ax the 
16 Study Sites: Entered Employment Raxes of Adult Terminees, Program 
Year 1988 



Site 



Entered mplc^ment rate 



Actual 



Prediaed 



Difference, 
in % points 



Fort Wayne, Ind. 
Coott Vtllcy. Ga. 
Corpus Chriiti, Tex. 
Jackson, Miu. 
Providence, R-I. 
Springfield, Mo. 
Jeney City. N.J. 
Marion, Ohio 
Oakland, Calif. 
Omaha, Neb. 
Larimer County, Colo. 
Heartland, FU. 
Northwest Minnesou 
Butte, Mont. 
Decatur, 111. 
Cedar Rapids, Iowa 

l&site average 
National average, 

all SDAs 

Source* Uowdihted aanual avences €alculil«d from JTPA Annual Sudu Repon (JASR) computer filet 
produced by the U.S. Depaitmeat of Labor. 

Ndes: Tht 'matd taf\aymm nte' it tbe percenUfC of all adult termittesi wbo had found a job before 
terainMiof tbeir ouollineat in JTPA. Tht 'predictod* «tei«d eaploymeat nu is based on tbe JTPA 
performance standaid icpoctod ia JASR, proffam year 19tS. 



0) 


(2) 


(3) 


84.0% 


72.4% 


ll.Olb 


83.5 


68.2 


15.3 


72.0 


67.1 


4.9 




69.2 


-1.6 


74.3 


70.2 


4.1 


89.0 


76.4 


12.6 


86.5 


64.2 


22.3 


55.5 


59.4 


-3.9 


67.4 


66.1 


1.3 


65.0 


65.7 


-0.7 


68.0 


69.5 


-1.5 


74.5 


68.7 


5.8 


73.5 


69.1 


4.4 


74.0 


67.1 


6.9 


79.4 


65.1 


14.3 


76.9 


73.1 


3.8 


74.5 


68.2 


6.2 


74.2 


67.3 


6.9 



recommended for different service strategies and consequently received different clusters 
of JTPA services. 



A Profile of the 1 8-Month Study Sample 

Hie 1 8-monih study sample comprises all experimental san^lemembers whoseFirstFollow- 
up Survey interview was scheduled for 1 8 or more months after their random assignment. 
Two-thirds of the 1 7,026 members of this sample wereintiie treatment group and one-third 
were in the control group. 



1 5. The only cxcq>tions ^^me treatment groi^ members excluded to maintain a constant 2/1 treatment- 
control group ratio. 
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ExhibU 3.9 Agency Performance Standards and JTPA TuU II-A Performance at the 16 Study Sues: 
Positive Termination Rates and Entered Employment Rates of Youth Terminees, 
Program Year 1988 



Positive termination rate Entered emplaymau rme 









Difference, 






DiJfererKe, 




Actual 


Predicted 


in % points 


Actual 


Predicted 


In 7c poinis 


Site 


(IJ 




wJ 




fS) 


(6) 


Fort Wayne, Ind. 


11% 


75% 


2% 










n/a 


n/a 


n/a 


48 


41 


7 




78 


72 


6 


48 


48 


0 


JtcksoD, Miss. 


/o 




4 


34 


44 


-10 


Providesce, R.I. 


75 


78 


-3 


54 


46 


8 


Sprin^idd, Mo. 


94 


76 


18 


70 


56 


14 


Jcney City, N.J. 


85 


80 


5 


n/a 


n/a 


n/a 


Marion, Ohio 


74 


75 


-1 


44 


38 


6 


Oakland, Calif. 


73 


78 


-5 


50 


45 


5 


Omaha, Neb. 


81 


73 


8 


n/a 


n/a 


n/a 


Larimer County, Colo. 


72 


74 


.2 


n/a 


n/a 


n/a 


Heartland, Fla. 


77 


74 


3 


49 


35 


14 


Nortfaweic Minnesota 


76 


78 


-2 


38 


44 


-6 


Butte, MonL 


86 


76 


10 


56 


45 


11 


Decatur, Ul. 


74 


74 


0 


25 


10 


15 


Cedar Rapids, Iowa 


66 


78 


-12 


60 


50 


10 


IC-site average 


78 


76 


2 


48 


41 


7 


National average. 














allSDAs 


81 


75 


6 


50 


41 





Source: Uowticbted avcvHO cakuktcd from HPA Aaaisl Sutui Report ^ASR) Modular fito 
oTI^xyr. 

Notca: Tbc 'poac^c ttrauoatiw nte* ti the percoa^ of «U yoi«h tcmuDCM 

foa^tjob.MtiieedftcocBixedca^kkjyaMMcoapeim eenp&eted doMOUiy, 

aecoe4aiy,crpo<t-aec'wSaiy Khod.caiQUed'>«KiUMrtiuBia(prQfrMaerstt W<^^ ealiMed k the Arned Fotdm, or 
racuiaed to Khod fulMiflft. llie *ttk«td )fCBtmc'Uthepwrcttce%fcoh>dfouad>job. Hie "pradided* me ef each of 
Ibeie ii bMed OB dbe mA pof oriiuBce «aadard Rpo^ 

a. bdudcibodiout-of-icboolndi]H«lMolyou*sstcsl4>21. Ttel t MtX udy tMapledocsaotiacladeia-iAodyfliahser 
yoUha uadcr ace 16. 

The sites' contributions to the sample ranged from 2,559in FortWayneto 346 in Cedar 
Rapids, as shown in the first column of Exhibit 3.10. Target group composition varied by 
site (columns 2 through 5), because of diiSerences in both the eligible populations and the 
recruiting and service emphases across sites, as well as certain exclusions firom the study. 
Most noiably, youths in Oakland were excluded from the study at the request of the site. 
The size of the youth target groups at the other study sites also reflects differing emphases 
on serving in-school versus out*of*school youths. Because in-school youths were not 
induded inthestudy,those sites thattargeted muchoflheiryouth program onlfa^ 
have a lower percentage of youths in their study sample. 

Exhibit 3.1 1 shows samplesizes in thefourstudytarget groups — adult women and men, 
andfemaleand male out-of-school youths — ^andthreeservicestrategysubgroups: classroom 
training, OJT/JS A, and othe*- services. Adult women formed about 39 percent of the sample 
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Exhibit 3.10 SampU Sizes in the 18-Month Study: FuU Sample and Target Groups, 
by Study Site 



Site 



FuU 
sample 
(1) 


Adult 
women 
(2) 


Ad^y 

men 

(3) 


Female 
youths' 
(4) 


MaU 
youths' 
(5) 




937 


979 


312 


331 


1,806 


788 


407 


410 


201 


1,498 


524 


412 


335 


227 


1375 


410 


398 


321 


246 


1,277 


376 


388 


229 


284 


1,202 


401 


427 


191 


183 


1,170 


471 


298 


228 


173 


1,083 


421 


485 


90 


87 


1,043 


562 


481 


0 


0 


956 


512 


220 


150 


74 


668 


318 


234 


70 


46 


597 


234 


202 


93 


68 


498 


163 


224 


55 


56 


477 


183 


138 


71 


78 


471 


177 


219 


30 


45 


346 


130 


114 


57 


45 


\lfi2JS 


6,607 


5,626 


2,649 


2,144 



Fort Wayne, Ind. 
Coosa Vall^^Ga. 
Corpus Chiistiy Tex. 
Jackson, Kiiss. 
Picvideoce, R.I. 
Springfield, Mo. 
Jers^ City, N J. 
Marion, Ohio 
Oakland, Calif . 
Omaha, Neb. 
Larimer County, Colo. 
Heartland, Ra. 
Northwest MinnesoU 
Butte, Mont. 
Decatur, 111. 
Cedar R#pid«, Iowa 

All sites 



Source: UMdjuttod frequcncici bwcd on B^ckaround Infotmatioo Foim rMpooiet. 
a. Out-of-tcbool youths only. 

(6,607 out of 1 7,026); adult men, a third of the sample; and out-of-school youths about 28 
percent(with 15.6percentfemaleand 12.6percentmale). Withinthefull sample SDAstaff 
recommended about 36 percent for the classroom training strategy, 38 percent for the 
OJT/JSA strategy, and 26 percent for the other services strategy. 



Assignees, Enrollees, and Control Group Members 

\s explained in Chapter 2, this report presents two sets of impact estimates: impacts per 
JTPA assignee (treatment group member) and inferred impacts per JTPA enrollee (treat- 
ment group member who became enrolled in JTP A after random assignment). Exhibit 3.1 2 
shows 63.8 percent of thetrealment group becameenroUedinJTPAat some pointduring 
the 18-monthfollow-upperiod. EnroUment rates acrossthefourtargetgroups varied by only 
a few percentage points, but differed more substantiaUy among the service strategy 
subgroups. Ineverytargetgroupthehighestenrollmentwasamongthoserecommendedfor 
theclassroomtrainingstrategy.foUowed by theother services and OJT/JSAstrategies,inthat 

order. 
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Exhibit 3.11 Sample Sizes in the 18-Month Stmfy: Full Sample and Target Groups, 


by Service Strategy Subgroup 










Full 


Adult 


AduU 


Female 


Male 


Service strategy 


sample 


women 


men 


youths' 


youths' 


subgroup 


(1) 


(2) 


(3) 


(4) 


0) 


Classroom tnining 


6,113 


2,927 


1353 


1,193 


640 


OJT/ISA 


6.410 


2322 


2.754 


612 


722 


Other services 


4,503 


1358 


1,519 


844 


782 


All subgroups 


17,026 


6,607 


5.626 


2.649 


2.144 



Source: Uoadjusted frequcnciec b«sed on Background InformaUon Form fcspooics. 
a, Out-of-acbod youths only. 



The variation in enrollment across the three service Strategy sub^^ 
As disciissed in more detail in the accompanying implementation report (Doolittle, 
forthcoming), the enrollment process was far from automatic. In the case of classroom 
training, for example, SDA stafif had toIinkprogramiq)pUcants withatrainingproviderthat 
would acceptthent For on-the-job training, ihestaffhad to find anemployer willing and able 
to offer the desired training in exchange for a subsidy. Enrollment ultimately occurred when 
staff entered an iq)plicant*s name into the local JTPA management information system, at 
which point this enroUee was counted among program participants for the purposes of 
meeting JTPA performance standards (see Appendix F for more detail). 

Exhibits 3.13 (for adults) and 3. 14 (foryouths) display selected baseline characteristics 
ofthesample,withseparatecolummforthecontrol group and JTPAassignees and cn^ 
Within each target group there were only very small differences among tiiese three groups. 



Exhibit 3.22 TreatTnent Group Enrollment in JTPA: Full Sample and Target 
Groups, by Service Strategy Subgroup 



Service strategy 
subgroup 


Full 
sample 
0) 


AduU 
women 
(2) 


AduU 
men 

(3) 


Female 
youths' 
(4) 


MaU 
youths' 

0) 


Classroom trmining 


72.4% 


72.8% 


112% 


ns% 


74.8% 


OJT/JSA 


565 


55.4 


56.6 


SIS 


S8i 


Other services 


623 


62.4 


58.9 


63.1 


67.7 


All subgroups 


63.8 


64.6 


60.8 


6SS 


66.8 


Sample size 


11A74 


4.465 


3.759 


1.814 


1.436 



Source: EnfoHment and ti«cking <Uu from the 16 lervice delivety jtbm (SDAs). 



«. Out-of-*cbool yoiaht only. 
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Exhibit 3J3 Seleaed Baseline Charaaeristics of Adults in the Sample: Control Group and 
JTPA Assignees and Enrollees, by Gender 



Characteristic 




Adult women 






Adult men 




Controls 


Assignees 


Enrollees 


Controls 


Assignees 

(5) 

LL? 


Enrollees 
(6) 
















▼* itHJCf iimi xuvjMuzxv 


53.6% 


54.5% 


56.2% 


57.9% 


57.0% 


57.6% 




31.1 


30.8 


27.9 


29.2 


28.8 


27,6 




12.3 


11.4 


12.2 


9.1 


9.7 


10.3 


A m^ncflfi Indi An or AlaaVa n Wfttiv6 


1.1 


1.0 


1.0 


1.3 


1.6 


1.2 


Asian or Pacific lalander 


1.8 


2.3 


2.6 


2.6 


2.9 


3.3 
















Receiving caah welfare 






17 0<K 

^ / .7 7v 




14.6% 


14.2% 


No high achool oiploma or 














G£D certificate 












zy. / 


Worked leaa than 13 weeks 














in past 12 months 


53.3 


53.8 


52.6 


40.6 


.5 


40.9 


Number of bamers 














None of the above 


27.6 


28.6 


30.1 


39.7 


37.7 


39.1 


One of the above 


34.2 


34.5 


34.4 


39.2 


41.1 


40,6 


Two of the above 


28.4 


27.8 


27.0 


16.9 


17.9 


17.2 


All three of the above 


9.8 


9.2 


8.4 


4.2 


3.4 


3.2 


Work and training histories 














Ever employed 


85.2% 


85.6% 


86.2% 


91.6% 


91.3% 


91.2% 


Mean individual earnings 














in past 12 months 


$2,352 


S2,362 


S2,386 


$4,093 


S3,948 


S4,011 


Hourly earnings in most recent job 














Never employed 


14.8% 


14.4% 


13.8% 


8.4% 


8.7% 


8.8% 


Less than $4 


34.2 


34.3 


35.1 


18.2 


19.8 


20.3 


$4 or more 


51.0 


51.3 


51.1 


73.4 


71.5 


70.9 


Employed upon ^plication 


14.6 


14.6 


15.0 


13.8 


13.1 


12.7 


Previously received occupational 














training 


45.2 


46.1 


46.7 


43.7 


47.2 


47.6 


Public assistance status ^ 














Receiving any public assistance 


61.9% 


58.7% 


58.0% 


37.8% 


37.0% 


36,1% 


Receiving AFDC 


34.8 


33.8 


34.6 


5.8 


6.0 


6.6 


Receiving food stamps 


53.3 


50.3 


49.6 


28.7 


28.7 


28.5 


Receiving other public assistance 


20.3 


18.4 


18.2 


19.9 


18.9 


17.9 



(Continued) 
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Exhibit 3.23 Seleaed Baseline Characteristics of Adults in the Sample: Control Croup and 
JTPA Assignees and Enrollees, by Gender (continued) 



Adult women Adult men 

Controls Assignees Enrollees Controls Assignees Enrollees 



Characteristic (J) Q) (J) (4) (5) (61 

AFDC history 



Never AFDC case head 




AO 1 C 




01 1 4Ct 




on 1% 






23.1 


24.1 


7.3 


6.9 


7.6 


AFDC case head 2 yeani or more 


30 1 


27.9 


27.2 


1.6 


1.7 


1.7 


J 1 r A reQuiiTu lor wciiaref looa 














SuunpSy or VTin pra^am 




12.3% 


11.4% 


7.8% 


9.6% 


8.2% 


Household oomposkioD 














No spouse or own child present 


21.3% 


CAT 

22.5% 


20.7% 


CO oar 
38.2% 






Own child under age 4, 














no spouse, present 


21.6 


20.1 


20.4 


3.7 


3.0 


3.2 


Own child, none under 4, 














no spouse, pirs^v 


ft 


34.9 


35.8 


5.1 


4.6 


5 0 


Stkoute Dfesenl with or 














without own child 


21.3 


22.6 


23.2 


33.0 


35.8 


37 0 


Family iacome in past 12 months 














< $3,000 


30.5% 


30.8% 


29.5% 


34.2% 


35.8% 


35.5% 


$3,000-56,000 


35.4 


33.7 


33.6 


26.6 


25.1 


24.4 


$6,001 - $9,000 


16.7 


16.1 


16.6 


16.2 


16.2 


16.3 


> $9,000 


17.5 


19.4 


20.3 


22.9 


23.1 


23.8 


LiTing Li public bousing 














Yes 


11.9% 


11.8% 


11.3% 


7.5% 


6.3% 


5.5% 


No 


88.1 


88.2 


88.7 


92.5 


93.7 


94.5 


Age at random assignment 














22-29 


43.4% 


44.2% 


44.0% 


43.2% 


44.6% 


45.5% 


30-44 


44.6 


43.1 


44.0 


45.1 


43.3 


42.3 


45-54 


7.9 


8.0 


7.4 


7.9 


7.8 


7.8 


> 54 


4.1 


4.7 


4.6 


3.9 


. 4.3 


4.5 


Mean 


33 J 


33.3 


33.2 


33 J 


33.1 


33.0 


Sample size 


2,142 


4,465 


2,883 


1,867 


3,759 


2,286 



Source: Uaidjimcd Ireycaciw b— ed on Backf rouad Iafoc»Mi.j« Foro napcmatm. 
a. AFDC, GcaenJ Attiiuaoe, or other welfare cxoepc food Aaopi. 

k. *Aay public MtifUttoe* udutk* the followtof toum of wiiatinnf AFDC, food fUmpt. UACflployacH inainocc, koutwf 

aMifiMoe, and oiber cmIi AMfUooe. 
e. *0(ber public MtiUMoe* iadudci uacoployaMt inaiifiiirf, boutia| MaifUMce^ ftod oiker (m^-AFDC/ caah ■Mi a tMcc . 
4. WINtitkefedonlWocicIacMtivepfOfnA. 
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Exhibit 3. 14 S^Uaed Baseline Characteristics of Om^f-School Youths in the Sample: 
Corarol Group and JTFA Assignees and EnroUees, by Gender 



Female youAs 



Male youths 



Omtrcls Assignees EnroUees 



Oiaracteristic 



Ethnidty 

White, non-Hispmic 
Bladr, non-Hispanic 
Hispanic 

Amcricsa Indian or Alaskjui Native 
Asian or Pacific Islander 
BAniers to cmployincnt 
Receiving cash welfare 
No high achool diploma or 

GED ceitificate 
Worked leas than 13 weeks 

in past 12 months 
Number of barriers 

None of the above 

One of the above 

Two of the above 

All three of the above 
Work and traming histories 
Ever employed 
Mean individual earnings 

in past 12 months 
Hourly earnings in most recent job 

Never employed 

Leas than $4 

$4 or more 
Employed upon application 
Previously received occupational 

training 

Public Msistaatt tUtuc ^ 

j^eceiving any public assisunce 
Receiving APDC 
Receiving Tvod «amps 
Receiving other public assistance ^ 



m 


a) 


0) 


47.4% 


50.1% 


51.2% 


34.7 


32.5 


28.2 


16.4 


15.8 


18.8 


0.7 


0.8 


0.9 


0.7 


0.8 


0.9 



Controls Assignees Enrollees 
(4) (5) (6) 



30.8% 30.3% 30.1% 



52.4% 

31.5 

14.6 

0.6 

1.0 

9.6% 



53.7% 

29.5 

14.7 

1.1 

1.0 



55.2% 

26.5 

15.8 

1.2 

1.2 



11.0% 11.4% 



49.0 


49.2 


52.5 


57.6 


59.1 


61.7 


59.4 


60.5 


59.9 


49.5 


47.0 


48.7 


21.8 


20.8 


21.3 


2Z3 


23.9 


21.8 


32.2 


32.3 


30.0 


41.4 


40.9 


40.4 


31.3 


32.3 


32.9 


32.3 


29.5 


32.3 


14.7 


14.6 


15.9 


4.0 


5.7 


SJS 



77.3% 


76.9% 


76.8% 


86.0% 


83.9% 


83.7% 


$1,384 


$1,305 


$1,341 


$2,114 


$2,071 


$2,007 


22.8% 


23.1% 


23.2% 


14.0% 


16.1% 


16.3% 


47.0 


47.6 


49.0 


36.5 


34.9 


35.5 


30.2 


29.3 


27.9 


49.5 


49.0 


48.2 


14.3 


14.4 


15.6 


11.9 


11.4 


12.1 


25.7 


25.5 


23.8 


31.1 


29.7 


29.6 


48.4% 


47.1% 


46.3% 


26.6% 


29.5% 


30.8% 


27.5 


26.6 


26.6 


4.4 


6.2 


5.7 


40.0 


39.3 


39.6 


22.2 


25.0 


26.0 


10.1 


10.5 


9.6 


10.2 


11.0 


11.8 
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ExhibiJ 3.14 Seleaed Baseline Characteristics of Ous-ofSchool Youths in the Sample: 

Control Groiq> and JTPA Assignees and EnroUees, by Gender (continued) 



Characteristic 



Female youths 

Controls Assignees Enrollees 
(2) (2) (3) 



Male youths 



Controls Assignees En rolUes 
(4) (5) (6) 



AFDC history 
Never AFDC cue ..ead 
AFDC caie head less than 2 years 
AFDC case head 2 years or more 

JTPA required for wdfare, food 
sumps, or WIN progranf^ 

Household composiUoa 
No spouse or own child present 
Own child under age 4, 

no spouse, present 
Own child, none under 4, 

no spouse, present 
Spouse present, with or 

without own child 

Family income In past 12 months 
< $3,000 
$3,000 -$6,000 
$6,001 ' $9,000 
> $9,000 

Living In public housing 
Yes 

No 

Age at random assignment 

16-19 
20-21 
Mean 
Sample size 



73.5% 71.0% 
20.3 21.9 



6.2 



34.5 
1.3 
12.4 



11.3 
15.7 

15.1% 
84.9 



7.2 



8.1% 7.3% 
51.8% 52.5% 



34.7 

1.5 
11.4 



45.5% 45.8% 

27.5 28.5 



9.8 
15.9 

13.9% 
86.1 



63.2% 59.4% 

36.8 40.6 

28.9 29.0 
835 2.824 



70.8% 
22.2 
6.9 

7.1% 
51.0% 
35.2 

1.6 

12.2 

46.5% 
27.7 
10.0 
15.8 

13.8% 
86.2 

59.3% 
40.7 
29.0 
2,288 



98.3% 
1.7 
0.0 



0.2 

10.1 

40.8% 
27.2 
12.1 
19.9 

12.3% 
87.7 



97.8% 
1.9 
0.2 



97.5% 
2.2 
0.2 



5.5% 5.9% 5.7% 
85.7% 85.2% 84.1% 
4.0 3.9 3.8 



0.4 

10.5 

39.7% 
26.3 
11.3 
22.8 

10.7% 
89.3 



59.2% 62.1% 

40.8 37.9 

29.0 28.9 

708 2,436 



0.4 
11.8 

40.4% 

25.7 
11.8 
22.1 

10.7% 
89.3 

63.9% 
36.1 
28.9 
959 



Source: Unsdjuitad frequencies bseed on Bsckgrouiid Infonratioo Focm i 
«. AFDC, Genenl AMi«ance, or cOxtr welfare except food Mnpt. 

b. *Any public aMitfsace* includes the Ibllowii^ sources of «Mi«sncc: AFDC, food eumpt, unemployment insurtnce, bouanf 
•mimmc, end other csih SMiiunce. 

c. 'Other public sMAsnce* iocludee unemploymem ioiunnce, bouwig SMiMancc, tod other (ooo-AFDQ csih saeittsace. 

d. WIN UUiefedenlWoric Incentive prpgnun. 
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The similarity between control group members and JTP A assignees was expected, given 
the strict random assignment process that created the two groups. Indeed, a detailed 
statistical comparison of the treatment group (assignees) and the control group reported in 
Appendix A, found no systematic or statistically significant differences m baseline charac- 
teristics. The similarity between assignees and enroUees is also noteworthy. If the sites 
encountered special difBculties in arranging services leading to enrollment for a pamcdar 
subgroup the assignees in that subgroup would be less Ukely to enroU in JTPA than other 
subgroups, and wewould then see differences inbaselinecharacteristicsbetweenassignees 

and enroUees. Thai appears not to have been tiie case. 



Target Groups 

ExhibitsS 13and3.14alsoservetodistinguishamongthefourtargetgroups. Theadultsin 
the sample tended to be young; around 45 percent were under age 30. and only around 12 
percent were age 45 or older upon their appUcation to JTPA. Adult men tended to be the 
most employableand to havethemostextensivework experience. MorespecificaUy.nearly 

70percentofthemenhadahigh school diplomaorGeneralliducationalDevelopment(GED, 
or high school equivalency) certificate; and men also had thehighest average earnings of the 
four groups and were the most likely to have earned more than $4.00 hourly in their most 
recentjob. 

Adult women andfem-aleout-cf-schoolyouths were considerably more likelythantheir 
male counterparts to be receiving public assistance and to have limited recent work 
experienceOessthan 13 weeksintheyearprecedingtheirappUcation). They were also much 
more likely to have a dependent child in the household. 

The youth target groups were the most disadvantaged. Only about half of all female 
youths and 40 percent of the male youths had a high school credential. Youths' average 
earnings in the year before their application were only slightly over half the earnings level 
of their adult counterparts, and they were much more apt to Uve in families with less than 
$3,000 in income over tlie previous 12 months." 



Subgroups Facing Selected Barriers to Employment 

To determine whether JTPA impacts varied with the degree of labor market 
disadvantagedness,thel8-month analysis includes separateestimatesfor subgroups facing 
selected barriers to employment. Following a framework developed for several studies of 

16. DifTercnccs in the baseline characteristics of the three service strategy subgroups are examined 
in chapters 4, 5, and 6. 



123 



JTPA 18-MONTH IMPACTS / CONTEXT»63 



JTPAby theU.S. General AccountingOffice(1989, 1990, and 1991), this analysis focuses 
on the following barriers: 

• welfare receipt, measured as receiving Aid to Families with Dependent 
Children (AFDC), General Assistance, or other cash welfare upon 
application to JTPA; 

• limited education, measured as a lack of a high school diploma or GED 
certificate upon application; and 

• limited recent work experience, measured as having worked less than 13 
weeks in the 1 2 months preceding the application. 

The incidence of each of these barriers among control group members, assignees, and 
enrollees is shown in the second main panel of Exhibits 3. 13 and 3. 14. Exhibit 3.15 shows 
the proportion of the treatment group overall and in each target group facing none, one, two, 
orallthreeof these barrierstoemploymcnt Almost 70 pcrccntof fliefullsamplewasfacing 
at least one barrier, but only 6.5 percent was facing all three. Hiere were important 
differences among the target groups, however. As noted earlier, Ae out-of*school youths in 
the sample were the most disadvantaged; here they exhibit iht highest incidence of 
the barriers, withfemaleyouthsatthe extreme with 42.0perccntfacingtwo or Aree barriers. 
Adult men were at the other extreme, wiA 80.8 percent facing none or only one of the 
barriers. 



Exhibit 3. 5 Distribuxion of the Treatment Group among Subgroups Facing a Certain 
Number of Barriers to Employment: Full Sample and Target Groups 



Number of barriers 
to employment 


Full 
sample 

(1) 


AduU 
women 
(2) 


Adub 
men 

(3) 


Female 
youths' 
(4) 


Male 
youths^ 
(5) 


None 


31.6% 


29.6% 


40.3% 


22.9% 


25.9% 


1 


38.4 


36.6 


40.5 


35.0 


42.4 


2 


235 


26.1 


I6J 


29.6 


26.8 


3 


6.5 


7.6 


2.9 


12.4 


5.0 


Sample size 


11.474 


4.465 


3.759 


1/814 


1.436 



Source: Uiiadjutted frequencies b*sed on BftckcftMind InfonnAtioo Fonii fetponaes. 

Noie: FerceoUgee mty not turn to lOu percent becAUic of rounding. The bairien to caploymeac ue the 

receipt of c««h wdftre (AFDC, Generil Aifii««nce, or o*her welfare except food fltampt), having no high 

ichool diploma or GED ceniftcate, and having worked Icat than 13 weeks in the 12 months prior to 

application to JTPA. 

a. Out-of-school youths only. 
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Patterns of JTPA Enrollment and Service Receipt 

Thissectionprovidesacontextforunderstandingtheimpacts estimated in the later chapters, 
by describing the service strategies JTPA staff recommended for the sample, the services 
in which each service strategy subgroup enrolled, and the duration of their enrollment. 



Service Strategies Recommended 

After samplemembers applied to JTPA,aSDAsta£f member assessed their interests,skills, 
serviceneeds, and personal preferences for the purpo.;^ of recommending an appropriate 
combination and sequence of program services. The personal preferences of the applicant 
were often an importantfactor in recommendingaparticularclusterofservices, since JTPA 

is a voluntary program and many individuals would not participate in services that did not 
interest them. For example, some applicants with Umited education wanted ajob rather than 
any classroominstruction, and this preferencemighthavelead the intake staffto recommend 

services such as on-the-job training or job search assistance, even though the staff would 
otherwise have recommended basic skills training. 

As explained in Chapter 2, we categorized sample members by the three clusters of 
services, orservice strategies, thatSDAstaff advised us reflected the pattern ofservicesthej' 
recommended. Exhibit3.16 shows that the service strategy most often reconunended was 
O JT/JS A, which SD A staff recommended for 37 .4 percent of the treatment group overall. 
TTiis was closely followed by classroom training, recommended for 35.9 percent; other 
services was the choice for 26.7 percent 

The aggregated figures mask distinct differences in service recommendations by gender 
and age, as shown in the columns of the exhibit for the target groups. 

Exhibit 3.16 Service Strategies Recommended for the Treaxmeta Croup: 
Full Sample and Target Groups 



Recommended 
service strategy 


FuU 
sample 
(1) 


Adub 
women 

a) 


Adult 
men 

(3) 


Female 
youths' 
(4) 


Male 
youths' 
(5) 


Classroom trsuning 


35.9% 


44D% 


24.6% 


443% 


29.9% 


OJT/JSA 


37.4 


35j0 


48.7 


232 


325 


Other seivices 


26.7 


21J0 


26.7 


32.5 


373 


Sample size 


11.474 


4.465 


3.759 


1.814 


1.436 


Source: Untdjusted frequcncict based on Background Infonnttion Fonn re«pontct. 
a. Out-of-school youths only. 
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• Females, both adults and out-of-school youths, were more likely than males to 
be recommended for the classroom training strategy. Among adults, 44.0 
percent of the women, as opposed to only 24.6 percent of the men, were advised 
to pursue this strategy; among youths, the corresponding figures were 44.3 
percent versus 29,9 percent. 

• Males, both adults andy ouths, were more likely than females to be recommended 
forthe OJT/JSA strategy, Almosthalf of all men received 1his recommendation, 
as opposed to only just over a third of the women. OJT/JSA was less often 
reconunended foryouths, but the gender difference was again substantial (32.9 
percent versus 23.2 percent). 

• Youths were more apt than adults to be recommended for the other services 
strateQK This strategy was advised for 37.3 percent of male youths and 32.5 
percentoffemaleyouthsbutforonly26.7 percent of adult men and21.0 percent 

of adult women. 



Differences in Employabiuty across Service Strategy Subgroups 

Throughout the later discussions of impacts onthethreeserviccstratcgy subgroups within 
each target group, it will be important to bear in mind tfiat tfie three subgroups differed fi^om 
one another in important ways. Afterall, SDAstaffmadefteir service strategy recommen- 
dations based in large part on each applicant's job skills and experience and needs for more 
or less intensive employment and training services. 

One clear w^tosummarize the differences amongthesubgroupsis to use estimates of 
thecontroIgroq>'seamingsoverthel8-mon<hfoUow-up period asanicasu^ 
Throughout this report the earnings of tfie control group serve as our estimate of what the 
treatmentgroupwouldhave earned in the absence of the program. Thus, they also indicate 
the treatment group's employability without access to the program. 

Exhibit 3.17 displaysthe total 18-month earnings ofcontrol group members in various 
target group-service strategy subgroup combinations. Differences in employability across 
subgroups are readily apparent. 

In eveiy target group exceptadultmen, those controlgroup members reconunendedfor 
ihtOJT/JSA strategy had the highest averageeamings over the 18 months following random 
assignment. Among men the earnings of the other services subgroup were slightly above 



IT These estimates arc baaed on the First Follow-«p Survey daU used in the analysis of program 
impacts in the later chapters. 
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Exhibit 3.1 7 Mean Earnings of the Comrol Group over the IS-Momh Follow-up 
Period: Full Sample and Target Groups, by Service Strategy 
Subgroup 



Mean earnings of the control group 



Service strategy 
subgroup 


Adult 
\vomen 

(1) 


Adub 
men 

a) 


Female 
youths" 
(2) 


Male 
youths" 
(4) 


Classroom mining 


$6^1 


$ 11,780 


$5^36 


$ 9,783 


OJT/ISA 


8,607 


12,456 


7,620 


12,765 


Other services 


7,960 


12,516 


5,726 


9,839 


AU subgroups 


7,488 


12306 


6,225 


10,736 


Sample size 


2.142 


1.867 


835 


708 



Source: E«tinuic« btied on Hni Follownip Survey icspontet. The citinutc* for adult women mc alto 
based on camingf data ftom itate unemployinent insuimnee (UI) agencies. 

Note: Eitimatc* aie legieaiion-adjuitcd to control for diffeiencet in baseline chaimcteristics between the 
treatment group and control group; see Appendix D. 
42. Out-of-school youths only. 



those of the OJT/JSA subgroup, probably because almost a third of the former group was 
job ready enough to be recommended for job search assistance only (not shown in the 
exhibit). Thelowestearningsamongadults,andhencetheleastjob-readysamplemembers, 
were in the classroom training subgroup. Incontrast,theleast job-ready youths appeared 
tobeconcentratedinboththeclassroomtrainingandlheotherservicessubgroup. Themost 
striking finding in the table is Ae high earnings level of male youth control group members 
recommended for OJT/JSA— which surpassed even that of adult men in the OJT/JSA 
subgroup. This estimate suggests that local staff routed a very job-ready group of male 
youths to OJT/JSA. 

Differences in JTPA Service Receipt across Target Groupt; 
AND Service Strategy Subgroups 

As noted in Chapter 2, the impact of JTPA depends on the difference in services received 
by those with access to the program and the services they would have received had they been 
excluded fi^om the program. Here we will simply summarize the JTPA services received by 
thetreatment group overall (including both those who did and those who did not enroll) and 
byenrollees only. 
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Exhibit 3.18 Receipt of Specific JTPA Services by the Treatment Group: 

Full Sample and Target Groups, by Service Strategy Subgroup 



% of treatment group receiving the service 



Specific program 
service 


FuU 
sample 

(I) 


Adult 
women 

a) 


Adult 
men 

(S) 


Female ^ 
youths 
(4) 


Male ^ 
youths 

(5) 






Qassroom training subgroup 




Never enrolled 


27.6% 


21.2% 


28.8% 


28.5% 


25.2% 


ClflMTOom training 
in occupational ildlls 


56.2 


57.8 


55.7 


54.8 


52.4 


Baaic education 


12.9 


10.6 


8.8 


n.8 


23.3 


On-the-job training 


3.8 


3.3 


5.4 


2.v5 


4.4 


Job aearch assistance 


19.5 


17.1 


12.4 


27.3 


30.8 


Work experience 


4.0 


3.9 


1.7 


5.7 


6.5 


Miscellaneous 


9.9 


11.3 


9.7 


7.7 


7.9 






OJT/JSA subgroup 






Never enrolled 


43.5% 


44.6% 


43.4% 


42.5% 


41.5% 


ClaMTOom training 












in occupational skills 


3.3 


5.1 


2.1 


3.3 


1.9 


Basic education 


3.1 


2.6 


3.6 


3.1 


2.8 


On-the-job training 


28.0 


28.5 


26.6 


29.9 


30.5 


Job search assistance 




26.5 


30.2 


28.3 




Work e3q>erience 


2.9 


2.6 


2.4 


5.2 


4.2 


c 

Miscellaneous 


6.5 


5.8 


6.8 


7.1 


6.8 






Other sendees subgroup 




Never enrolled 


37.7% 


37.6% 


41.1% 


36.9% 


32.3% 


Classroom training 












in occupation^ skills 
Basic education 


9.4 
15.7 


15.6 
11.1 


4.9 

6.1 


9.8 
29.7 


6.5 
26.4 


On-the-job training 


4.7 


5.5 


4.7 


3.9 


3.9 


Job search assistance 


19.7 


23.4 


24.8 


12.2 


12.0 


Work experience 


2.3 


2,7 


0.9 


3.4 


3.2 


c 

Miscellaneous 


31.0 


31.5 


28.4 


28.5 


35.3 


Sample size 


12,474 


4,465 


3,759 


1,814 


1,436 



Source: Earolincat sod tnduaf dau fioa tte 16 SDAa. 

Ovt-of-achool yoMiht ooly. 
h. "Basic educaiioa* iaclute Aduh Baw: Edactfioa (ABE), Ufk tdwol or Geaefal Eaucaboail 

Developmeai (GED) prepsmioa, sad Ea|liih ai a Seoead Lsafuaie (ESL). 
c. *Miicel]ftaeout* iacludei a MM saeet, job-readiacai tfaiatag » n>Mo«fT/>d tfiaaiaf » vocatioaal eiplontioa. 



job thadowifig, aad tiyout captoyneat, aaoag otker servioet. 



6S ♦ JTPA l»-MOKTHIMPACrS / CONTEXT 



Exhibits. 18 details thespecific program servicesreceived by rrea/men 
in each service strategy subgroup within the full sample and the target groups. Recall that 
classroom training in occupational skills was the service defining the classroom training 
strategy and on-the-job training was the service defining the OJT/JS A strategy. The other 
services strategy, on the other hand, was defined as recommending neither of those services 
as the primary one. 

The patterns of service receipt were indeed quite different for the three subgroups. In 
every target group over 50 percent of the classroom training subgroup received the primary 
service — occupational skills training. 

Members of the OJT/JS Asubgroup, however, were ^t to receive one of two services: 
on-the-job training or job search assistance. The exhibit also shows that the service strategy 
definitions were accurate; only a small percentage of the classroom training subgroup 
received on-the-job training, and only asmall percentage of the OJT/JSA subgroup received 
classroom training in occupational skills. 

Exhibit 3. 19 focuses in on JTPA enroUees only and highlights the two key services 
received in each target group-service strategy subgroup combination. This breakdown 
provides fiirtherinsightintotheservicesreceived by the firsttwo service strategy 
and helps to untangle the more complex patterns of receipt for the other services subgroup 
inthe preceding exhibit Specifically: 

• Most of the treatment group members recommended for the classroom training 
service strategy received either classroom training in occupatioiud skills, the 
definingserviceforthat strategy, basic education, a secondary service, or both. 
Hence, in terms of the services actually received, it is most appropriate to 
characterize this service strategy as one focused on classroom instruction. 

• Most of the treatment group members recommended for the OJT/JSA service 
strategyreceivedeitheron-the-jobtraining, the definingserviceforthat strategy, 
job search assistance, asecondary service, or both. Hence, it is most ^propriate 
to characterize this service strategy as one focused mainly on immediate 
employment, with or without subsidized traininji,. 

• Most of the adult treatment group members recommended for the otfier services 
strategy received eitfier job search assistance or miscellaneous services, or both. 
Most of the youth treatment group members recommended for the other 
services strategy received either basic education, miscellaneous services, or 
both. Hence, adults in this subgroup were more likely to receive services that 
focused on immediate employment, whereas youths were more likely to receive 
services that focused on basic education and other preemployment services. 



JTPA IS-MONTHIMPACTS / CONTEXT • 69 



Exhibit 3, 19 Key JTPA Services Received by Treatment Group Members 

Who Were Enrolled in the Program: Target Groups, by Service Strategy Subgroup 



% of enroUees receiving one or both services 



Key services 
in each service 
strategy subgroup 


Adult 
women 

0) 


Adult Female^ 
men ywuns 
(2) (3) 


Male ^ 
youths 
(4) 






Qasjrocm training subgroup 




Clafisroom tnintng in 








occupational gldlls/ 
base education 


88.8% 


85.5% 86.1% 


80.4% 






OJT/JSA subgrcup 




On-th^-job training/ 
job search asaiatance 


87.8 


86.5 84.8 


84.5 






Other services subgroup 




Job aearch asaiatance/ 








mitcellaneoua 


82.3 


88.7 




Basic education/ 








miscellaneous 




79.5 


83.2 


Sample size 


2,883 


2,286 2,288 


959 



Source: EoroUmetu tod ttackiog dau from the 16 SDAft. 
Nou: Atrfiowni&theboUompaoel.tbefceyacrvioMieccivvdbytheoU^ 
daffettd between adulu sad out-cf-ecbool youths. 
Out-of-ecboo) youths only. 

*BMie educatioo' iadudcs Adult Buic Educstton (ABE), hi^ echool or GeoenI EAic at io oa l 
Devclopmett (GED) preparedoo, sad E^gliifa as a Sacood Uqfusfe (ESL). 
e, 'KCicelUmous' iDcIudee aueeanetit, job-fcadinees tfaioiag, cuttomczed tniouig, vocatiooal cxplonKioo, 
job ahadowiqg, and txyoul dqploymeift, arooof other lefvicct. 

The findings for classroom training are strai^tforward, butthose for the OJT/JSA and other 
services strategies require some background on program operations and the research design 
to be clearly understood. 

TheOJT/JSAservicc strategy was intended for applicants seeking employment ^o, in 
the judgment of local stafi; appeared to need on-the-job training and a wage subsidy 
to developtheskiilsnecessaiytobehiredasunsubsidizedworkers. The initial actions taken 
to arrange on-th&-job training, however, were often very similar to what staff members would 
do to help an aqpplicant find unsubsidized employment; the first step in both cases was to find 
an employer interested in hiring a new employee. Furthermore, Ae ^>plicant might Hso be 
seeking an unsubsidized job, often with help fi^om SDA staff in tiie form of job search 
assistance. As shown earlier in Exhibit 3.18, across all target groups roughly equal 
proportions of treatmwit group members (ranging firom 26.5 percent to 32. 2 percent) who 
had beenrecornmendedforlheOJT/JSAstnrtegy subsequently cnroUed in on-the-job tr 
or job search assistance. Thus, in each target group, members of the OJT/JSA subgroup 
were just as likely to receive JS A as OJT. 
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The other services strategy, by definition, involved a diverse group of clients. Under the 
research design, SDA staff recommended this strategy for ^plicants facing serious 
employment barriers, who needed basic education or preemployment skills enhancement 
before they could benefitfirom classroomtraining in occupational sldUs or on-the-job training, 
or before they could be expected to land ajob. The strategy wa: also deemed ^propriate 
for those needing specialized services, such as a customized combination of classroom and 
on-the-job training, vocational exploration, job shadowing, and tiyout employment, amonga 
large number of other services that varied across sites. These specialized services are 
groupedtogeAeras**miscellaneousservices**inthecxhibitsof thisreport Finally, the other 
services strategy was also ^propriatc for those who wereso obviousfy employable that they 
needed only job search assistance. Althou^ job search assistance was a comnion activity 
in all three service strategies, it was only in the other services strategy than an ^plicant could 
be recommended for this service alone. Thus, within the other services subgroup the 
difference in tiie service pattern between addts(who received primarilyjob search assistance 
or miscellaneous services) and youths (who received primarily basic education or miscella- 
neous services), as shown in Exhibit 3.19, suggests that those very job-ready applicants 
recommendedforthis strategy wereprimarily adults, whereas theyouthsrccommendedfor 
this strategy tended to have more serious skill deficits that had to be addressed first through 
basi c education or preemployment skills training. 



Enrollment Patterns Over Time Among Enrollees and 
THE Treatment Group Overall 

There were also clear differences among the service strategy subgroups and, to a lesser 
extent, targetgroupsinthedurationoftheircnrollmcntinTitlell-A. AsshowninExhibit3.20, 
among treatment group members who were enrolled in JTPA those recommended for the 
OJT/JSA strateg>' tended to have the shortest periods of enrollment, whereas those 
recommended for the classroom training strategy had the longest The relatively short 
average enrollments among members of the OJT/JSA subgroup reflect their higji rate of 
receiving job search assistance, a service that tends to be very brief There were also 
differencesamongthetargetgroups,evcnwithinservicestrategies. Adultmalecnrolleeshad 
the shortest period of program participation overall, and in both the classroom training and 
other services subgroups. 

Another vfzy to characterize the services received by the treatment group is to identify 
a period during which most of its members were enrolled in the program and aperiod during 
which most were no longer enrolled. Exhibit 3.21 presents the proportion of each target 
group-service strategy subgroup combination enrolled inTitle II- A in various months after 
random assignment There was asharp drop in enrollment between the third and sixA month, 
and by the s . cth month less than 1 5 percent of both the OJT/JSA and the other services 
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Exhibit 3.20 Median Number of Months Enrolled in JTPA among Treatment Group 
Members Who Were Enrolled: FuU Sample and Target Groups, by 
Service Strategy Subgroup 



Service strategy 
subgroup 


Full 
sample 
(I) 


Adult 
women 
(2) 


Adult 
men 
(3) 


Female 
youths" 
(4) 


MaU 
youths" 
(5) 


Classroom trr^aing 


5.0 


5.6 


3.8 


55 


4.6 


OJT/JSA 


2.0 


1.9 


2.1 


2.0 


2.2 


Other services 


2.6 


2.4 


1.7 


33 


2.9 


All subgroups 


3.3 


3.6 


25 


3.9 


3.1 


Sample size 


7J16 


2^ 


2086 


1J88 


959 



Soutce: Enrollment tad timcking data from the 16 SDAt. 
a. Out-of-tchool youths only. 



subgroup was enrolled. Thcservices received by the classroom trainingsubgroup obviously 
lastedlonger. Fnrolhnent rates of the classroom training subgroup dropped to 15 percent or 
below sometime betweentheninth and twelfth month for treatment group members overaU, 
but not until between the fifteenth and eighteenth month for adult womea 



JTPA Services Received by Treatment Group Nonenrollees 

The previous subsections focused on the program services received by treatment group 
members overall and by those who were enrolled in the program. But because the inferred 
estimates of program impacts per JTPA enroUee in this report require the assumption that 
treatment group nonenrollees experienced no JTPA impact, it is important to review 
briefly the extent to v^toch nonenrollees did or did not receive JTPA services. This review 
also provides additional ]n.';ightinto program operations. 

ToinvestigateAeissue, the implementationresearch team conductedaseparate analysis 
of JTPA service receipt by a small subsample of those treatment group members >^iio did 
notbecomeenrolledintheprogramduringthe 18-monthfollow-up period. (See Appendix 
F and, for more detail, Doolittle, forthcoming.) This analysis found that SDA staff members 
worked to some extent with approximately half of these treatment group nonenrollees, 
primarily in attempts to arrange services by, for example, referring them to potential 
employers for on-the-job training, providing job search assistance, or attempting to arrange 
classroom training. The remaining half either lost interest in the program or found another 
training or employment opportunity on their own. 
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Exhibit 3.22 Treatment Group Enrollment in JTPA in Seleaed Months of the 
18'Month Follow-up Period: Full Sample and Target Croups, 
by Service Strategy Subgroup 



Percentage of the treatment group enrolled in JTPA 



Month 




Full 
sample 
(1) 


AduU 
women 

(2) 


AduU 
men 

0) 


Female 
youths" 
(4) 


Male 
youths' 
(5) 








Classroom training subgroup 




Month 


3 


56.8% 


58.8% 


49.1% 


61.0% 


56.4% 




6 


35.8 


39.6 


26.9 


382 


33.1 




9 


23.7 


27.0 


16.6 


253 


20.0 




12 


14.9 


173 


103 


13.9 


145 




15 


13.9 


16.0 


10.4 


12.7 


142 




18 


13.1 


152 


9.4 


12.6 


12.6 








OJWSA subgrotq> 






Month 


3 


283% 


26.6% 


29.1% 


29.9% 


292% 




6 


123 


122 


122 


13.5 


133 




9 


62 


6.7 


5.4 


7.4 


6.4 




12 


3.9 


A3 


33 


4.0 


3j6 




15 


3.9 


4.6 


3.4 


4.0 


3.6 




18 


3.4 


42 


23 


3.6 


3.8 








Other services subgroup 




Mondi 


3 


33.9% 


302% 


27.0% 


42.7% 


43.7% 




6 


14.0 


143 


11.9 


15.8 


153 




9 


7.0 


9.8 


6.1 


63 


4.7 




12 


4.8 


72 


3.9 


4.4 


3.0 




15 


4.6 


6.8 


3.6 


4.1 


32 




18 


4.0 


62 


3.3 


3.1 


2.4 


Sample size 


11.474 


4.465 


3.759 


lfil4 


1.436 



Source: EnroIImeiit and tncking <ktft from the 16 SDAs. 
a, Out-of-icbool youths only. 



In general, nonenrollees for whom staff tried to arrange OJT or classroom training clearly 
received much less intensive JTPA services than those treatment group members who 
actualtywereenrolledinprogramservicesandbegantoreceivetraining. On fte other hand, 
those referred to job search assistance who never became enrolled mxy have participated 
in much fte same services as those who did ultimately enroll; but the nonenrollees were 
generally less successful in finding employment. Overall, about 82 percent of the treatment 
group (the 64 percent who enrolled plus abouthalf of the 36 percent who did not enroll) had 
some contact with JTPA, of widely varying degrees of intensity, after their random 
assignment. 
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Two final points about these services to nonenrollecs should be made. First, it is likely 
that the JTPA performance standards system is a major reason why local JTPA staff wait 
until applicants actually begin receiving aformal classroom training or an on-the-job service 
before enrolling them in the progran As explained earlier in this chapter, the Department 
of Labor assesses the performance of each SDA based on the success of its program 
terminees, as measured based on their employment rates, wage rates, and — ^for youths — a 
broader measure called **positive termination.'* But only those individuals ^o formally 
enroll in JTPA are counted in this performance standards system, and so the SD As have an 
incentive to wait until an iq)plicant actually begins receiving a service designed to increase 
employability before enrolling him or her. In many SDAs initial assessment, counseling, 
development of a employability plan, and referrals to potential service providers all typically 
happen before applicants are fonnally enrolled in JTPA. 

Second, this pattern of services provided to nonenrollees does not bias the estimates of 
program impacts pera55ignee reported in Chapters 4 through 7, since these measure the 
impact of access to JTPA on all members of tiie trcatmwit group, vdiether they become 
enroUedornot Furthermore, the benefit<ostanalysisthatwiU appear in ourfortfi 
report will include estimates of the JTPA costs of serving all those in the treatment group; 
and thus the comparison of impacts per assigr^ee with costs per assignee necessary for Aat 
analysis will also be valid. Thefindingthatsome JTPAservices wereprovided to treatment 
group nonenrollees is cause for caution, hov;ever, in interpreting the alternative impact 
estimates — ^impacts per enrollee — ^presented in the following chi4)ter5. As explained in 
Chapter 2, impacts per assignee should be interpreted as a lower bound, and impacts per 
enrollee, as an upper bound, on the true impact of JTPA on those treatment group members 
who actually received some program service. 

Summary 

The most basic conclusion of this chapter is that the study sites and the 1 7,026 members of 
the Ig-monthstudysampleresembleSDAsandtheirparticipants nationally and also include 
much of their diversity. The sites, though not chosen randomly, include several with very 
strong economies during the late 1 980s, others experiencing modest growth, and still others 
slowlyrecoveringfromjob losses in therecessionoftheearfy 1980s. Furftermore, the sites* 
performancecnkcyDepartment of LaborstandardsforTitlefl-Aprogranisshowed similar 
diversity, with both strong and weak performers, as defined by those standards included in 
thestudy. As with tiie sites, the members of thestudy samplearetiiemselves adiverse group, 
allowing an analysis of program impacts ster^iming fi'om each of the three JTPA service 
strategies as well as impacts on numerous subgroups of interest to policymakers and program 
planners. 
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The diversit> among the Study sites in labormarket conditions, in the characteristics of 
the people they served at the time, and in service duration and costs has a second clear 
implication forthe analysis: We can expect to see wide variation in program impacts across 
thesites,buttheveiyfactof that diversity will makeitdiflficdt to explain the differences 

precisioa 

In addition, the study* s definition of three main service strategies recommended by SD A 
staff did produce distinctsubgroups that differed systematicallynotonly in the types of JTPA 
services they ultimately received but also in their baseline characteristics. As a result, the 
estimates of program impacts presented in later chapters wiU offer an assessment of how 
well these clusters of services — classroom training, on-tiie-job training^ob search assistance 
(0 JT/JS A), and other services — were working for the particular groups of applicants they 
were designed to serve. 

This dizpler has also provided a context for understanding the JTPA services both 
recommended for and received by treatment group members— the source of the program 
impact Just undertwo-thirds(63.8perccnt)of tiietreatmentgroupwascnrolledinJTPA 
at some point in the 18-month follow-up period for this report In general, the services 
enrollees received were of a relatively short duration, reflecting JTPA's national emphasis 
during the 1980s on job placement and low servicecosts. Those treatment group member 
recommended for the classroom training strategy were the most likely of the three servia 
strategy subgroups to be enrolled in JTT A, to stay enrolled for arelatively long time, and to 
receive some form of classroom instruction. Those recommended for tfie OJT/JSA 
strategy had the lowest overall enrollment rates, but they were also the most employable 
of the three groups; enrollees here tended to receive on-the-job training, job 
search assistance, or both. As intended, the other services subgroup included individuals 
with veiy diverse backgroimds and service needs. Some adults in this group were also 
among the most employable, and so a substantial number of them were enrolled in job 
search assistance only, whereas youths in this group tended to be among the least job-ready, 
ard so they tended to be enrolled in basic education and other services intended to address 
their lack of work experience. 



Adult Women: JTPA Impacts at 1 8 Months 



THIS chapter presents our findings on the impacts of JTPA Title II-A on the earnings, 
employment, and high school attainment of adult women at the 16 study sites. These 
impact estimates provide the first reliable evidence of the effectiveness of Title II-A for adult 
women. Moreover, they constitute the first major experimental evaluation of the effects of 
employment and training programs on women not receiving Aid to Families with Dependent 
Children (AFDC), who make up two-thirds of the women in our study sample. 

The 6,607 adult women in this 18-month study sample were ages 22 or older at their 
random assignment, and like the adult men reported on in the next chapter, they had an average 
age of 33, with about 88 percent between ages 22 and 44 and less than 5 percent age 55 or 
older.^ The women resembled the men in several other respects: over half were white, 3 1 
percent were black, and 1 1 percent were Hispanic. Over two-thirds had a high school 
credential upon application to JTPA, and most (86 percent) had worked before, although few 
women or men were employed when they applied to the program.^ 

Unlike the men, however, nearly half of the women had worked less than 13 weeks in the 
past 12 months, and over a third had earned low wages (less than $4.00 an hour) in their most 
recent job. Thus, the average earnings of adult women in the year before their application to 



1 . The estimated impacts on earnings and employment reported in this chapter are based on the 5,826 
women who responded to the First Follow-up Survey (88.2 percent of the 18-month study sample) and on 64S 
nonrespondents for whom earnings data from state unemployment insurance agencies were available . Appendix 
D describes the procedures used to combine the respondent and nonrespondent data. 

2. The information on baseline characteristics presented in this and the next paragraph is based on 
Background Infomiation Fonn responses from all women in the 18-month study sample. For more detail see 
Exhibit 3.13 in Chapter 3. 
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JTPA ($2,359) were much lower than those of adult men ($3,996). As might be expected, the 
women were also much more likely to have a child of their own in the household (70 percent 
of the women as opposed to 33 percent of the men) and to be receiving public assistance, 
including food stamps (60 percent of the women versus 37 percent of the men). As a resuk 
of aU these differences, the women in the sample were probably less readily employable than 
the men. 

This chapter begins by presenting the estimated average impacts of JTPA on adult women 
overall. The first main section examines impacts on earnings, and the second, impacts on 
employment. Using the findings from these two sections, we then decompose the estimated 
impacts on earnings into effects (»the percentage employed, weeks worked if employed, hours 
worked per week worked, and earnings per hour woriced— what we term the components of 
earnings. The estimates in this third sertion offer insight into such issues as whether the 
program led to more stable jobs, more fiiU-timc jobs, or jobs paying higher wage rates. 

The fourth and fiifth sections focus on certain subgroups of women of interest to 
policymakers and program planners. The fourth section looks at wwnen recommended for 
each of the three broad clusters of program services we have defined as service strategies: 
classroom training, on-the-job training/job search assistance (OJT/JSA), and other services. 
The estimated impacts on earnings by service strata subgroup offer evidence of the effects 
of different clusters of JTPA activities on the subgroups of women whom program staff 
deemed ^propriate for each cluster. 

In the fifth section we turn to oihef selected subgroups, defined by characteristics, such 
as barriers to employment fiicing women, that may have affected their abUity to benefit from 
the program. Here we also present estimates that control for the distribution of each key 
subgroup across sites and for its distribution across both sites and service strategies— to help 
distinguish between effects across subgroups that reflect differences in the way subgroups are 
distributed across sites or service strategies and those that reflect differences in the 
characteristics of the women themselves. 

In the ch;q>ter summary we review the findings, with reference to results from previous 
experimental studies of the effects of employment and training programs serving women. 

The impact estimates presented throughout this chapter were derived by the methods 
outlined in Ch^ter 2. However, unlike the estimates for adult men and youths in the following 
chapters, these for adult women were adjusted for survey ncmresponsc bias, because our tests 
for nonresponse bias using earnings data fr<xn state unemployment insurance (UI) records 
indicated a small but nonncgligible bias in the impact estimates for this target group. For a 
fiill discussion of the methodologies employed in all these procedures, see Appendix D. 
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Impacts on Earnings: Adult Women Overall 

This section presents estimates of program impacts on the earnings of adult women. These 
estimates serve as summary measures of the overall effects of JTPA Title II-A on this target 
group and as a benchmark for comparing the estimates for subgroups of women presented 
later in the ch^tcr. 

This section begins by contrasting the monthly earnings trends of treatment group and 
control group members over the 1 8*month follow-up period bcginmng at random assignment. 
It then describes the pattern of the treatment group's cnroUment in JTPA over the same time 
period, to distinguish between the in-program and post-program periods for the group as a 
whole. The section proceeds by detailing our estimates of program impacts on assignees and 
enrollees and ends by examining impacts on the distribution of ea rn i ng s. 

Monthly Earnings Trends: Treatment Group and Control Group 

Exhibit 4 . 1 shows the earnings txperience of treatment group and control group members over 
the first 1 8 months after their random assignment.^ As shown, the treatment group— ^osc 
women in the sample who had access to JTPA— consistentiy had higher average monthly 
<^miri£g than the control group^ whose members were denied access to the program for the 
purposes of the experiment. 

The estimates underlying the f^™^z^ curve for the treatment group represent an outcome 
of JTPA — what the treatment group earned after its members gained access to JTPA: from 
$268 in the first follow-up month to $521 in the eighteenth.^ This information does not tell 
us what the treatment group would have earned without access to JTPA, which is what we 
must know to calculate the impact estimates we will present shortly. 

It is the control group curve that provides this information. Average earnings of control 
group members ranged from $261 in the first follow-up month to $478 in the eighteenth.^ 



3. Throughout this chapter, earnings and impact estimates are expressed in nominal dollars. The follow- 
up period varied across indi viduals» beginning as early as November 1 987 and ending as late as December 1 990. 

4. We used ordinary least squares r^ression procedures to increase the statistical precision of these 
estimates, as described in Appendix D. 

5. The earnings estimates shown in Exhibit 4.1 and subsequent exhibits include wages paid to JTPA 
participants in on-the-job training positions. During the 18-month follow-up period the program rdmburssd 
employers a total of about $650 per aduU female OJT participant. Among all adult women in the sample OJT 
reimbursements totaled about $80 per treatment group member over the 18-month follow-i^) period. 
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Exhibit 4.1 Average Monthly Earnings: Treatment Groiq> and Control Group 

Sl.OOO r 




Sourcet: E»tini«te« bued on Pint FoUow-up Survey retponie* and eamingt d«U from «Utc unemploy- 
ment insurance (UI) tgencie*. 

Note*: Sample tize. treatment group » 4.376. control group « 2.098. EitimatM are rcgreif lon-adjutt 
to control for difference* in baaeline characteristic! between treatment group and control group; «ec 
Appendix D. 



These point estimates actually provide two crucial pieces of informaticm. First, in 
representing our estimates of wiiat the treatment group would have earned, on average, 
without access to JTPA, they serve as the basis for the treatment-control group comparisons 
of the experimental impact analysis . And second, they demonstrate the importance of making 
these cOTiparisons in evaluating employment and training programs, since they indicate that, 
even without access to JTPA, control group members increased their average monthly 
earnings substantially over the course of the follow-up period. 

Control group members were excluded from JTPA services only; they were free to 
participate in any non-JTP A employment and training services available in the community. 
The control group earnings curve shown in Exhibit 4. 1 therefore reflects the effects of any 
non-JTPA services received by control group members. Thus, the benchmark against which 
we measure the impact of JTPA services is not the total absence of any employment and 
training services but the level of non-JTPA services available in the community. That is, we 
measure the incremental effect of JTPA, over and above the effects of other available 
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employment and training services.^ (Later in this chapter we present estimates of 
received by both treatment group and control group members.) 



Enrollment Patterns over Time: Treatment Group 

In the early part of the follow-up period the dififerential between treatment group and control 
group earnings may have been affected by the treatment group's participation in JTPA, in 
several different ways. On the one hand, enroUmcnt in occupational skills training or basic 
education may have actually delayed employment for some treatment group members early 
in the fc!low-up period, causing their earnings to be lower than those of their control group 
counU .parts. On the other hand, CTTolhncnt in job search assistance or on-the-job training 
may have led to fester job plfxcments for other treatment group members, causing their 
earnings to surpass those of their control group counterparts in the early months of the follow- 
up period. 

Any differential between treatment group and conUol group earnings later in the follow- 
up period, however, should be relatively free of these conflicting effects of program 
participation and should therefore reflect the post-program effects of JTPA. Exhibit 4.2 
allows us to establish this distinction between in-program dnd post-program periods, by 
showing the percentage of all those assigned to the treatment group who were enrolled in JTPA 
Title n-A in each month of the follow-up period.^ 

Among adult women enrollment rates of the treatment group dropped from 58 percent in 
the first follow-up month to only 11 percent in the eighteenth.' Throughout this report we will 
use the first month in which less than 15 percent of the treatment group was still enrolled in 
JTPA as the beginning of the post-program period; thus, the post-program period for adult 
women began in month 10. The estimated impacts for the last eight months of the follow-up 
period may therefore be viewed as primarily post-program impacts. By referring back to 
Exhibit 4. 1, we can see the positive treatment-control group dififerential in earnings persisted 
well after most treatment group members had left the program. 



6 . In the bcncfit-cost analysis to be included in the final report, we will also measure the inaemental costs 
of JTPA, for comparison with these mcremental program impacts. 

7. Exhibit 4.2 is based on daU from SDA records that somewhat ovcnrtatc the number of persons still in 
the program at any given time because the data are missing some tcmiination dates. Thus, this graph serves as 
an upper bound on the percentage of the treatment group still in the program in any given month; and the extent 
to v^rhich the graph ovcrsUtes the actual enrollment rates is higher in the later months. 

8. Although 65 percent of the adult female treatment group was enrolled in JTPA Title II-A at some point 
during the follow-up period, only 58 percent was enrolled in the first follow-up month, because some treatment 
group members enrolled later. 
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Exhibit 4,2 Percentage Enrolled in JTPA Monthly: Treatment Group 

100% r 



80% 




0% ' ' ^ ^ " • ' ^ ^ ^ ^ ^ ^ ^ ^ ^ . . , 

0 3 6 9 12 15 18 

Months after random assignment 

Source; Unadjusted frequencies bated on enrollment and tracking dau from the 16 service delivery 
areas (SDAs). 

Note: Sample size» treatment group ^ 4,465. 



Impacts ON Earnings: JTPA Assignees and Enrollees 

In the simplest terms our estimates of average program impacts on earnings equal the 
difference between the average earnings of treatment group members and the average 
earnings of control group members . Exhibit 4.3 serves to illustrate our derivation of these 
estimates. 

First note that we have shifted from monthly to quarterly earnings. The first column in 
the exhibit shows estimates of the average quarterly earnings of the treatment group, wiiich 
rose from $942 in the first foilow-up quarter to $1,555 in the last, and average total earnings 
over all quarters in the follow-up period, which were $8,027. Column 2 shows the 
corresponding cstinutcs for the control group, which rose from $916 in the first quarter to 
$1,414 in the last, for atotal of $7,48S over the fiiU period. 

Column 3 presents the estimated impacts* the differences between columns 1 and 2. 
Significance tests indicate that these differences were statistically significant in the last five 
quarters of the follow-up period and for the period as a whole (as indicated by the asterisks 
by these estimates). This means we have confidence that JTPA increased the earnings of adult 
women in these time periods; in other words, it is unlikely that these treatment-control group 
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Exhibit 4,3 Impacts on Earnings: Treaiment Group 

Impact on 

Mean earnings treatment group earnings 



Treatment Control In ^ ^ 

group group s of a) 

Period (J) O) (S) W 

Qiuiter 1 $"9^^ S 916 5^26 l9? 

2 1,205 1,145 60* 5.2 

3 1,353 1,236 118«* 9.5 

4 1,442 1,363 78« 5.8 

5 1,529 1,413 116«* 8.2 

6 1.555 1,414 141«* 9.9 
All quarters ZfiT? 7^488 539*** 7.2 



Sources: Euimaies baied od Fim FoUow-up Suivey rapooM aad etnings 6mm {torn *Mc UI afeocies. 
Noi£s: Sample «zc, tssigoee* » 4,376; conirol j roup « 2»098. EAimates arc rcgre*sioD-adju«ed lo comrol 
for differaces io baseline chanctcrisiics between Ibe treatmeoi group aad cooirol group; see Appendix D. 
♦ Suiiaically significam M the .10 level. at ibc .05 level, ai tbe .01 level (two-iailed te«). 
A. Significaace levels for this column arc identical lo tbote in column 3. 

differences in earnings are due to random sampling error. The estimated effect over the 1 8- 
month period was an earnings gain of $539. Column 4 expresses the dollar estimates in 
column 3 as a percentage of the control group mean. For simplicity's sake we do not show 
significance levels in this column, since they are the same as those in the corresponding rows 
of column 3.^ 

The impact estimates presented in columns 3 and 4 of Exhibit 4.3 measure the average 
effect of assignment to the treatment group on all treatment group members, regardless of 
their participation in JTPA. We therefore refer to these estimates as impacts per JTPA 
assignee. These estimates provide the most direct, reliable experimental evidence of the 
impacts of JTPA and, as such, are the estimates on which we will focus throughout this report. 

But as shown in the previous chapter, not all JTPA assignees (treatment group members) 
actually enrolled in JTPA. It would therefore be of interest to know the impact of the program 
on only those who did enroll. Our estimates of impacts per JTPA enrollee adjust the estimated 
impacts per assignee to account for the fact that 35 .4 percent of adult female treatment group 
members did not become enrolled in JTPA and for the fact that 2.9 percent of adult female 
control group members did become enrolled (despite the experiment's embargo on their 
participation).*^ 

9. To increase the staiisiical precision of these estimates, we used ordinary least squares regressions. 
This reduced the standard errors of the impact estimates but d' ^ not appreciably aficct the poinl esiimalcs. 
because the average values of the independent variables (mainly ihe baseline characteristics of the ircalmeni 
and control groups) were virtually the same for the two groups. Sec Appendix D for a full description of these 
procedures. 

10. Appendix D details the procedures used for these adjustments. 
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The adjustment for the treatment group's enrollment rate requires that we assume there 
was no JTPA impact on those members of the ueatment group who did r.ot formally enroll m 
the program. If some nonenroUees did experience positive effects from the program, the 
estimates of impacts per enrollee would ovcrstote the impart on enrollees, because the 
adjustment would attribute these nonenroUee imparts to the enrollees." 

We explore this issue in Appendix F, which presents the results of a special study of JTPA 
services received by a subsample of nonenroUed treatment group members. These findings 
suggest that roughly half of all adult female nonenroUees in the 18-month study sample 
received some form of JTPA service after their random assignment, in most cases job search 
assistance or referral to an employer for a possible on-the-job training position. Since these 
services were limited in scope and intensity, their impacts on nonenroUees were probabK' 
negligible. Nevertheless, wc cannot be sure of this conclusion, and there is evidence to 
suggest the validity of this conclusion may vary among women recommended for different 
service strategies. 

We therefore consider our inferred imparts per JTPA enrollee to be estimates of the 
upper bound on the size of the average program impart on enrollees, since they may spread 
the total impart over too few treatment group members— that is, only those who formalh' 
enrolled. In contrast, we consider our estimates of impacts per JTPA assignee to be estimates 
of the lower bound, since they spread the total impact over too many treatment group 
members; that is, they include some assignees who had no exposure to the program beyond 
the act of application. 

Exhibit 4.4 displays the estimated impacts on quarterly earnings per assignee and per 
enrollee. The first three columns simply repeat the information about impacts per treatment 
group member (assignee) presented earlier in Exhibit 4.3. The first column shows estimated 
average earnings of the control group. Columns 2 and 3 repeat the dollar and percentage 
impacts per assignee. Column 4 presents the estimated impacts per enrollee, which are 
uniformly larger thcjx the estimates per assignee by a factor of 1 .62.'^ 

As shown in the exhioit, the inferred impacts on the earnings cf adult women who enrolled 
in JTPA ranged from $42 in the first quarter after random assignment to $228 in the sixth, for 



11 It is possible that the program impact on nonenrolled treatment group members was the opposite 
of any impact on the enrollees. If this were so, which seems unlikely, the estimated impacts per enrollee 
would understate the magnitude of the average impact on enrollees. 

12 As explained in Appendix D, the adjustment factor used to derive impacts per enrollee from impacts 
per assignee is l/(r-c), where r is the enrollment rate (the proportion of treatment group members who 
enrolled in JTPA) and c is the crossover rate (the proportion of control group members who were enrolled 
in JTPA) Since these two rates are fixed for any given group or subgroup, the ratio of impacts per emollec 
to impacts per assignee is also fixed for any given group or subgroup. Thus, for example, for the adult female 
target group the impact per enrollee is 1 .62 times the impact per assignee for all outcomes m all time periods. 
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Exhibit 4.4 Impaas on Earnings: JTPA Assignees and EnroUees 





Control 


Impact per assignee 


Inferred impact per 




mean 


InS 


As % of(l) 


enroUee, in $ 


Period 


(1) 


(2) 


(3) 


(4) 


Quarter 1 


% 916 


$ 26 


2.9% 


$ 42 


2 


1,145 


60* 


5.2 


97 


3 


1.236 


118*** 


9.5 


190 


4 


1,363 


78** 


5.8 


127 


5 


1,413 


116*** 


8.2 


188 


6 


1,414 


141*** 


9.9 


228 


All quarters 


7,488 


539*** 


7.2 


873 



Source*: E«tiiii*ie« baaed on Ftrat Folkmnip Survey reipoiuea «x»d ctrntvi daU from aute UI agenciea. 
Notea: Sample «ze, aaaigneea m 4,376; control fro^) « 2,098. Eatimatea are r^reaaioo-adjuated to cootrt4 
for differeacea In baaeline characteriatica betweeo the treatmeat group and control group; aae Appendix D. 
• Sutiatically Mgnificaat at the .10 level, at the .05 level, at the .01 level (two-tailed teat). 

Signficance levels for column 3 are identical to tboae in column 2. TeM of aUtiatical aignificance ^^ere 



not performed for column 4. 

atotal earnings gain of $873 over the entire foUow-up period. If treataient group nonem^oUe^ 
experienced ne^gible program effects, thetotal impact per enrollec was probably close to this 
upper bound estimate. But if nonenrollecs experienced a substantial positive cflfect from the 
JTPA sendees they received, the total impact per enrollee was probably closer to $539, the 
lower bound suggested by the total impact per assignee. Because of this uncertainty we did 
not attempt to calculate the statistical significance of the inferred inqjacts per enrollee in this 
column or elsewhere in this report. 



Impacts ON THE DisTRiBunoN OF Earnings: JTPA Assignees 

These average earnings gains for assignees and enroUees were not, of course, evenly 
distributed across all women. Some women may have gained substantially more than the 
average impact estimated, while others may have had small gains or even losses. We would 
like to know the distribution of program impacts on individual treatment group members. For 
example, does the $539 impact per assignee for the follow-up period as a v^ole rq)resent a 
large impact on only a small percentage of the women or modest impacts distributed across 
the sample? 

Because we cannot match up individual sample members in the treatment group with 
individuals in the control group, we cannot estimate the distribution of impacts on the 
earnings of individuals. We can, however, estimate program impacts onthe distribution of 
earnings of the treatment group. Here we are asking how JTPA changed the shape of the 
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Exhibit 4,5 Impacts on the Distribution of Total IS-Month Earnings: JTPA Assignees 



I8-month earnings 


Assignees 
(1) 


Control 
group 

a) 


Difference, 
in % poinu 
(3) 


$0 


21.0% 


23.5% 


-2.5 


$1 - $3,800 


18.7 


19.4 


-0.7 


$3,801 - $8,500 


19.1 


19.0 


0.1 


$8,501 - $14,300 


20.2 


19.1 


1.1 


> $14,300 


21.0 


19.0 


2.0* 



Chi-tquared test of impact on entire dittribtttion 



Soutcm: E«tiinit«i bMed oo Pint FoUwMip Survey retpooM* «Dd e«mngi dtU firom itMc UI «gendeft. 
Notfii: S»a^t itae, mngoM » 4,336; control ffoup •> 2,079. For the ettuntUob otocoAiPC, tec Appendix D. 
• Sutiitic«lly ligmfictnl at the .10 level, tt tte .05 level, at the .01* level (chi-«qu«od test or tvwt^led 
t-teet). 

earnings distribution. In other words, did the program increase the percentage of treatment 
group members with higher earnings or reduce die percentage with lower earnings? 

Exhibit 4.5 shows the distribution of total 18-nu)nth earnings for the treatment and 
control groups (columns 1 and 2) and the program impact on the proportion of JTPA 
assignees whose earnings over the period were within each earnings category (column 3). 
The first earnings category in the exhibit is defined as those sample members with zero 
earnings over the period; the remaining four cat^ories represent approximate quartiles of 
the earnings distribution of those control group members with positive earnings." 

As shown in column 3, which presents the differences between the distributions of the two 
groups, JTPA significantly reduced the percentage of assignees with zero earnings during the 
follow-up period by -2.5 percentage points. Tbis finding implies that the program increased 
the proportion of women yAko found jobs, which is consistent with the estimated impacts on 
employment presented in the next section. The program also significantly increased the 
proportion of assignees in the highest earnings category, those earning more than $14,300 
over the IS-mcwnth period. Unfortunately, we cannot tell where in the earnings distribution 
those assignees who would have had zero earnings in the absence of the program fell or where 
those whom the program moved into the top earnings category came from. Nevertheless, the 
test for the significance of the impact on the earnings distribution as a whole does tell us that 
the program did produce a change in the distribution. 



13. The method by which the estimated impacts on the distribution were adjusted for survey 
nonrcsponsc bias made it impossible to divide the control distribution into exact quartiles. See Appendix 
D for an explanation of fliis procedure. 
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Impacts on Employment: Adult Women Overall 

Although the estimated impacts on total earnings arc perhaps the best summary measures of 
the cflFect of JTP A Title II-A, it is also of interest to examine program efiTects on more detailed 
measures of labor maricet success. This section presents estimates of program impacts on 
three measures of employment — ^the percentage employed, the average number of weeks 
worked, and the average number of hours worked— dyixing each follow-up quarter and for 
the 18*month period as a whole. 

Exhibit 4.6 displays estimates based on these measures in three separate panels. The first 
panel indicates that JTP A significantly increased the proportion of adult women employed at 
some time during the thitd, fifth, and sixth quarters after random assignment and of those 
employed at some time during the follow-up period as a whole. Overall, the proportion of the 
treatment group ever employed during the follow-up period was 2. 1 percentage points higher 
than it would have been in the absence of the program.*^ Among JTPA cnroUees the 
proportion ever employed was an estimated 3.5 percentage points higher than it would have 
been without access to the program. 

The program did not have a statistically significant cflfcct on the average number of weeks 
or hours worked by assignees over the follow-up period as a wiiole. Although the estimated 
effects on average weeks (2.8 percent) and hours (3.7 percent) were roughly the same size as 
the effect on the percentage employed (2.8 percent), we cannot tcU whether there was a true 
impact on the average number of weeks and hours worked becaxise the impacts shown in these 
last two panels were estimated less precisely than those in the first panel.^^ 

There were, however, significant positive effects on weeks and hours worked in selected 
quarters . The average number of weeks worked was 4.2 percent and 4.9 percent higher among 
assignees than control group members in the third and sixth quarters, respectively; and the 
average number of hours were between 4.7 percent and 6.5 percent higher among assignees 
in the third, fifth, and sixth quarters. 

It is important to recognize that these estimated effects on the average number of 
weeks and hours of employment for all assignees may simply reflect the program's positive 
effect on the proportion of women employed (as shown in the first panel of the exhibit). 
Consider, for example, the following hypothetical case. Suppose that all women who 
would have worked without access to JTPA worked exactly the same number of weeks 



14. Because the outcome shown in the top panel of Exhibit 4.6 is the percentage of the sample rver 
employed in the relevant time period (quarter or IS-roonth follow-up period), quarterly proportions do not 
sum or average to the proportion for the follow-up period as a >^ole. 

15. These impactsareestimatcdlessprcciselybccauscweeksandhoursemployedhavehigher variance, 
relative to the estimated effect, than does the percentage employed. 



ERIC 



146 



«6 • JTPA Ig-MONTH IMPACTS / ADUIT WOMEN 



Exhibit 4. 6 Impacts on the Percentage Employed and on the Mean Number of 
Weeks and Hours Worked: JTPA Assignees and Enrollees 







Impaa 


per assignee 








In % pts.. 




Inferred impaa per 




Control 


weeks, or 




enrollee. in % pts. . 




mean 


hours 


As % of(l) 


weeks, or hours 


Period 


(I) 


(2) 


(B) 


W 






Percentage employed 




Quarter 1 


48.4% 


0.0% 


0.0% 


0.0% 


2 


53.4 


1.4 


2.6 


2.2 


3 


55.8 


2.4* 


4.3 


3.9 


4 


59.7 


0.8 


1.4 


1.3 


5 


59.3 


3.3*** 


5.5 


5.3 


6 


O0.9 


2.2* 


3.6 




Anytime dxiring 




2.1** 


2.8 


3.5 


quarters 1-6 


/U.O 






Weeks worked 




Quarter 1 


5.1 


0.0 


-0.2% 


A A 
U.U 


2 


6.0 


0.1 


0.9 


U.l 


3 


6.3 


0.3* 


4.2 


n A 

U.4 


4 


0.8 


0.1 


1.5 




5 


7.0 


0.3 


3.7 


0.4 


6 


6.9 


0.3** 


4.9 


0.5 


All quarters 


38.3 


1.1 


2.8 


1.8 






Hours worked 




Quarter 1 


183 


-1 


-0.5% 


-2 


2 


220 


3 


1.3 


5 


3 


232 


12* 


5.4 


20 


4 


250 


8 


3.0 


12 


5 


257 


12* 


4.7 


20 


6 


254 


17** 


6.5 


27 


All quarters 


1,403 


52 


3.7 


84 



Sources: Ectanutet batod on Hrtl FoUowntp Survey retporuet and eamiof • <Uu from tuic UI ^fMciet. 
Notet: Swnplc size, Mttgneet « 4,376; control group « 2.09S. Ettinutet are refrettioo-Adjuiled lo control for 
difTercacet in bateUne cbartcterittici between the treatment froup end control group; tee Appendix D. 
• SuUtUctUy tigmfictnt at the .10 level. tt the .05 level. tt the .01 level (two-t^W teet). Significenoe 
levels for column 3 •(« identical lo thoae in column 2, TeaU of tUtiitical tignificance ^re not performed for 



column 4. 
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and hours as they would have, but JTPA found employment for some women who would 
not have worked in the absence of the program. The additional weeks and hours of 
work associated with this latter group would raise thv^ average number of weeks and 
hours worked for all assignees, even though the program had no effect on the weeks and 
hours worked by those who would have been employed anyway. Here the impaa on 
average weeks and hours worked is simply a reflection of the program's impact on the 
proportion of all assignees who were employed at some tmie in any given period. 

In the next section we attempt to distinguish between the effects on average weeks 
and hours worked per assignee that merely reflect a highf;r average employment rate 
among all adult women assigned to JTPA and those that reflect additional weeks and 
hours of work for those women vAiO would havc^w^cd without access to the program. 
We do this by shifting our measures from average wedcs and hours worked by all 
assignees (including zeros for nonworkers) to average weeks and hours worked per 
week worked by only those women who did work. 



Impacts on the Components of Eamings: Adult Women Overall 

The analysis in this section provides more detailed measures of labor market success than 
those presented so far. Specifically, these measures enable us to estimate whether JTPA Title 
n-A led to more stable employment, to more full-time jobs, or to jobs that paid higher wage 
rates. 

The analysis is based on the fact that average eamings can be decomposed as follows: 

earnings workers ^ weeks ^ hours ^ earnings 

assignee assignee worker week hour 

Each of the four components of eamings in this relationship reflects a diflferent aspect 
of labor market success. Workers per assignee reflects the ability of assignees to find jobs— 
the "pure" employment effect. FFeeiks worW/^er sorter reflects both how quickly assignees 
found jobs and how long they held them, or the stability of their employment. Hours worked 
per week worked reflects the mix of workers' part-time, full-time, and overtime work, that is, 
whether they were more likely to fmd a full-time job. And earnings per hour worked reflects 
what workers were paid for the time they worked. 

Appendix D explains how we estimated JTPA impacts on each of these eamings 
components, and Exhibit 4.7 presents our findings. Because the percentage impacts on the 
last three outcomes shown in this exhibit were calculated indirectly, no tests of significance 
are shown in this exhibit. The significance levels of the impacts on to.al earnings (column 1) 
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Exhibit 4.7 


Percentage Impacts 


on Earnings and Its Components. 


JTPA Assignees 


Period 


Earnings 
per assignee 
0) 


Workers 

per 
assignee 
(2) 


Weeks Hours worked 
worked per week 
per worker worked 

(S) W 
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per hour 
worked 
(5) 


Quaiter 1 
2 
3 
4 
5 
6 


2.8% 
5.2 
9.5 
5.8 
8.2 
10.0 
7.2 


0.0% 

2.6 

4.3 

1.4 

5.5 

3.6 

2.8 


-0.3% 
-1.6 

0.0 

0.2 
-1.7 

1.2 

0.0 


-0.3% 
0.4 
1.1 
1.5 
1.0 
1.6 
0.9 


3.4% 

3.9 

3.9 

2.7 

3.3 

3.2 

3.4 



No««: Sunple lize, -.ipiee. - 4.376; oonuol r~F - 2.<»«. Eiumito Ke «fnMioD^*edU) comroJ lot 
aiffe«*w ia bMeUae durtcuaitaci betwoea the UtMrno* iwi^) tad 

Colum»»2lhioufb5diipUya»inip«*««pai»«Heof»heoon«^ f"' 
oolunm 2 thii mam the lOfMct oa the eaiployiBeat «te diipUyod u • pecc«u«e of the »e«i nie for the 
oootrol sroup. Tou of MiMtcid .if aificaee were >ot perfotmed for ary of the ooluaiu la thii eriiibit. 

and workers per assignee (column 2) are the same as those in exhibits 4.4 (for earnings) and 
4.6 (for percentage employed). 

Note that the percentage impacts on the four components in each row sum to approxi- 
mately equal the percentage impact on earnings in the same row. They therefore measure the 
relative contribution of each component to the estimated impact on earnings. 

As shown in the last row of the exhibit, the 7.2 percent average program impact on 
assignees' earnings overthefoUow-uppcriodasawhole (column 1) was primarily attributable 
to program impacts on the percentage of assignees who were cn^loyed some time during the 
period (column 2) and to impacts on the earnings per hour worked by those who were employed 
(column 5). Slightly under half of the overall impact on earnings was due to a 3A \ytrQCvx 
increase in hourly earnings while employed, and most of the remainder was due to an increase 
of 2.8 percent in the percentage who were employed. There was little or no effect on the 
average number of weeks worked by those who were employed or on the number of hours 
worked per week worked. Thus, JTPA appears to have increased the probabiUty that women 
wouldbe employed, and to have raised thcirhourly earnings when they did work, butitdid not 

seem to lead to steadier employment or a greater likelihood of full-time work. 

Caution must be exercised in interpreting the estimated impact on earnings per hour 
worked (column 5) since it may reflect program effects on the composition of the subgroup of 
women who were employed, in addition to— or instead of— a positive program impact on the 
hourly earnings of specific individuals. If, for example, the additional employment generated 
by the program was concentrated among women with high hourly earnings, the average hourly 
earnings calculated for all workers would increase even if the program had no effect on the 
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hourly earnings of any individual worker. Thus, the gain in average hourly earnings shON^-n 
in Exhibit 4.7 does not necessarily imply that JTPA increased the wage rates of individual 
workers. 

Similarly, program-induced changes in the composition of the subgroup of women who 
worked may also be masking cfifects on the average number of weeks and hours worked by 
individual women. Thus, the finding of Uttle or no effect on wedcs or hours worked by those 
who were employed should not necessarily be interpreted as evidence of a lack of effect on the 
weeks or hours worked by specific workers. 

The distinction between program effects on the composition of the subgroup that woiked 
and effects on specific individuals is especially important in the case of the estimates of 
earnings per hour worked, because increases in the hourly earnings of specific individuals 
may be evidence of a program effect on productivity. In an attempt to separate compositional 
effects fi'om effects on the hourly earnings of individual workers, we used noncxpcrimcntal 
estimation techniques to derive tiie impact of the program on "latent wage rates." 

A latent wage rate isthe wage rate that a worker could command in the market if she 
were employed. Since the latent wage rate is defined for both workers and nonworkcrs, 
estimated impacts on latent wage rates are fiec of any compositional effects that may be 
clouding our interpretation of impacts on the average observed wage rates of workers. 
Unfortunately, since the latent wage rates of nonworkers cannot be observed, they must be 
estimated with nonexperimcntal methods of unknown reliability. 

Our analysis of latent wage rates }s presented in Appendix G. The results of this 
analysis show no statistically significant effect of JTPA on the latent wage rates of adult 
women in the fifth and sixth quarters after random assignment, that is, during the post- 
program period when one would expecX any such effect to have materialized. For reasons 
discussed in the appendix, however, the nonexperimcntal methods used in this analysis may 
not be reliable. Thus, there may be no fully satisfactory way to address this issue with the data 
on the present sample. 

Impacts on Earnings, Employment, and Earnings Components: 
Adult Women Recommended for Each Service Strategy 

The impact estimates presented in the preceding sections arc averages for all adult women in 
the 18-month study sample. They therefore reflect the effects of JTPA Title H-A on a wide 
variety of individuals whom program staff reconunended for a number of different employ- 
ment and training services. This section focuses on the subgroups of women recommended 
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for each of the three dusters of services, or service strategies, defined in Ch^ter 2: the 
classroom training, OJT/JSA, and other services strategies. 

As we saw in Chapter 3, each of these service strategy recommendations resulted in a 
distinctly diflferent mix of services received by women in each of these subgroups. We would 
therefore expect that impacts might vary systematically among the three service strategy 
subgroups. It is important to bear in mind, however, that the three groups differed not only 
in the services each received, but also in the kinds of people within the group. Thus, any 
differences in in^acts across these subgroups cannot be attributed solely to differences in the 
kinds of services each subgroup received. In other words, oac cannot view differences in 
impacts among the service strategy subgroups as an indication of what might happen if a 
particular group of people were shifted from one strategy to another. Instead, the impact 
findings for the three subgroups must be viewed sq)aratfely— as indicating how well each 
strategy worked for the distinct group of people it actually served. 

We begin by examining how the employment and training services received by women 
varied across service strata subgroups. We then describe the differences in services 
received between the treatment group and control group within each subgroup. The cross- 
strategy comparisons recall the comparisons in Chapter 3 showing how the three subgroups 
differed on this dimension. The trcatment-amtrol group comparisons, on the other hand, 
reflect the increment in 5erwce5 attributable to JTPA— the 50«rceofthc impacts within 
each subgroup. As part of this discussion we also introduce estimates of program effects on 
the attainment of a high school diploma or General Educational Development (GED) 
certificate associated with participation in a school or training program. 

We then show how the three service strategy subgroups differed from oac another in terms 
of the kinds of women in each, as measured by their baseline characteristics . The remainder 
of the section presents the estimated impacts on each service strategy subgroup, based on the 
same measures of earnings, employment, and the ccmiponents of earnings as those in the 
preceding sections on adult women overall. 

Differences in Employment and Training Services Received: Treatment 
AND Control Groups wtthin Each Service Strategy Subgroup 

As described in Chapter 2, the three service strategy subgroups were defined based on the 
services recommended for thcni, not the services they actually received, because there is no 
way to identify the control group members who would have received a particular service. 
Because service reconunendations were made by program staff prior to random assignment, 
control group members can be matched to treatment group members on the basis of 
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recommended services. As a result, we can obtain purely experimental estimates of JTPA 
impacts on each of the three subgroups. 

Not all treatment group members received the services for which they were 
recommended. Among adult female treatment group members overall, 35.4 percent were 
never enrolled; by service strategy subgroup the percentages not enrolling were 27.2 percent 
for the classroom training subgroup, 44.6 percent for the OJT/JSA subgroup, and 37.6 
percent for the other services subgroup (Exhibit 3.12 in Chapter 3). And some of those who 
were enrolled received services other than the primary service recommended for them. 

Nevertheless, separating the sample on the basis of recommended services is useful to 
the extent that it creates groups that received distinctly different clusters of services, 
because it allows us to analyze the effects of different service strategies. As demonstrated 
in Chapter 3 (Exhibit 3.19), the three service strategy subgroups did receive distinctly 
different clusters of services. 

It must be recognized that each service strategy subgroup received a mix of services, 
not a single, isolated service. This means that the impacts reported for each service strategy 
subgroup reflect the effects of the mix of services received by that subgroup, not the impact 
of a single specific service. More specifically, as explained earUer, our impact estimates 
represent the incremental effect of the difference between the mix of services received by the 
treatment group and the mix received by the control group. 

As shown in Exhibit 3. 19 in Chapter 3, 88.8 percent of the adult female enroUees in the 
classroom training subgroup received classroom training in occupational skills, basic 
education, or both. Likewise, 87.8 percent of those cnroUccs in the OJT/JSA subgroup 
received on-the-job training, job search assistance, or both. Finally, 82.3 percent of cnrollecs 
in the other services subgroup received one or both of job search assistance or miscellaneous 
services, the key categories of services provided to adults recommended for this strategy. 

The impact of the program, however, does not dei^d solely on the JTPA services sample 
members received. Rather, the impact depends on^ difference between the services 
received by those with access to JTPA and the services they would have received if they had 
been excluded fi-om the program. That is, the counterfactual against which we evaluate JTPA 
is the services available elsewhere in the conununity, not a total absence of services. We 
represent the services treatment group members would have received without access to JTPA 
by the employment and training services received by the conuol group, whose members were 
excluded fr<m JTPA. 
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Exhibit4.8 displays estimates of this service difference for women recommended for each 
service strategy. The exhibit compares both the percentage of treatment group and conuol 
group members receiving each service (their likelihood of receiving the service) and the 
average number of hours of each service receivrd (the amount of the service received). 

Note that the estimates for different services were derived from different sources. There 
are several reasons for this. TixstyhccMSC classroom training in occupation ski lis md basic 
education are usually available from many non-JTPA providers, our only feasible source of 
information about these services was the First Follow-up Survey. But because survey 
respondents were likely to have difSculty distinguishing between JTPA-frmded instruction 
and instruction funded by other public and not-for<i)rofit programs, we did not attempt to make 
this distinction in the survey; we simply asked about employment and training services 
received from any provider. In general, however, we would expect most (but not all) of the 
services received by the treatment group to be JTPA services and virtually all of the services 
received by the control group to be non-JTPA services.^^ 

Second, we could not measure receipt of on-the-job training or work experience in the 
survey, because resp<»idents were likely to have difficulty distinguishing these positions from 
regular jobs. We therefore used enroUment and tracking data from the 16 JTPA service 
delivery areas (SDAs) to estimate receipt of these services by both the treatment group and 
the control group. Since these two services are usually not offered by non-JTPA providers, 
tfid^eatment-control group difference in receipt of these services is probably a reasonably 
reliable indicator of the difference in service receipt. 

A more serious omission results from the frict that the survey could not measure receipt 
of Job search assistance over the course of the fiiU follow-up period. The research team 
assumed that respondents simply would not be able to recall their receipt of this service — 
which is usually both brief and nonintensi ve — as long ago as 1 8 months before the interview. 
And since many ccmtrol group members were likely to receive job search assistance from non- 
JTPA providers, the treatment-control group difference in receipt of this sendee from JTPA 
only would not be a reliable indicator of a service receipt difference. Finally, we also did not 
collect survey data on the services classified as miscellaneous, because ofbotfa potential recall 
problems and the difficulty of defining these services, which were much more numerous and 
varied than the examples listed in footnote c of the exhibit. 

In summary, then, we were able to obtain reasonably complete survey data on receipt of 
classroom training in occupational skills and basic education from any service provider, for 



16. The expecution regnrding the treatment groiy is based on a comptfison of the survey-based data and 
the enrollment and tracking data from the 16 service delivery areas reported in Chapter 3. The expectation 
regarding the control group is based on the fact that only 2.9 percent of the adult female control group enrolled 
in JTPA during the 18-mon1h follow-up period. 
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Exhibit 4,S Receipt cf Empkyment and Traimng Services: Treatment Group and Control Group, 
by Service Strategy Subgroup 
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both the treatment group and the control group. We were also able to obtain SDA data on 
receipt of on-the-job training and work experience from JTPA providers, which serve as a 
fairly rehable source for the treatment-control group OHnparisons. But for receipt of job 
search assistance and miscellaneous services we report estimates only for the treatment group, 
because there was no reliable source of information on the control group's receipt of these last 
two categories of program services. 

Before turning to the findings in Exhibit 4.8, note that the last three columns measure 
treatment-control group differences in the amount of services received as the average number 
of hours received by all sample members in the subgroup, including those who did not receive * 
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Exhibit 4. 8 Receipt of Employment and Training Services: Treatment Group and Control Group, 
by Service Strategy Subgroup (continued) 
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services. To calculate an estimate of the average number of hours of receipt for only those 
sample members vAto actually received a service (service recipients), simply divide the hours 
for all treatment or control group members in the subgroup (column 4 or 5) by the proportion 
of the treatment group or control group members in that subgroup who received the service 
(column 1 or 2). Finally, also note that one cannot sum the percentages receiving services in 
each column because individual sample members might have received more than one service. 

The Classroom Training Subgroup* Although all women in the classroom training 
subgroup were recommended for classroom training in occupational skills, only 48.6 percent 
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of the treatment group reported receiving this service, as shown in the first panel of 
Exhibit 4.8." More than a quarter of the corresponding control group reported receiving this 
service. Neither of 4esc outcomes should be surprising. 

Within the treatment group the principal reason vAxy some women did not receive the 
services recommended for them was that 27 percent never enrolled in JTPA after gaining 
access to the program (sec Exhibit 3. 12 in Chapter 3). As noted at 4e start of this discussion, 
we expected that a number of control group members would receive employment and training 
services; by virtue of 4eir application to JTPA, control group members demonstrated their 
motivation to seek out these services. In the case of the classroom trpining subgroup, the 
estimate for the control group in column 2 of the exhibit means that over a quarter of the 
control group succeeded in finding instructional services from other providers. 

In fact, most communities have numerous providers of occupational skills training and 
basic education, many of which are subsidized by public fiinds. For example, community 
colleges offer a wide variety of vocational and technical courses at heavily subsidized tuition 
rates. And the same federal student aid programs that JTPA often taps to help support its 
trainees are available to students taking courses on their own, at either conunimity colleges 
or private technical schools. State vocational education programs provide a variety of courses 
on a subsidized basis as well, and local public school systems and community-based 
organizations offer basic education classes for low or no fees. 

Thus, with respect to occupational skills training and basic education, JTPA is not ^e 
only game in town.*' The estimates in&e first panel of Exhibit 4.8 demonstrate, however, 
that ^ program did increase the rate of receipt of these services. In the classroom training 
subgroup treatment group members were more likely tiian the control group to receive 
classroom training in occupational skills (48.6 percent vs. 28.7 percent) and were also 
somevsiiat more likely to receive basic education (1 1 .2 percent vs. 7.5 percent). 

In the classroom training subgroup, the average difference in the amount of service 
received between all members of the treatment group and the control group (column 6) was 
110 additional hours for classroom training in occupational skills and 9 additional hours for 
basic education— that is, additional hours of service receipt attributable to the treatment 
group's having access to JTPA. 

These relatively modest treatment-control group differentials in the amount of classroom 
training received per assignee reflea two factors noted earlier. First, not all treatment group 



17. The survey-bAsed estimates of receipt of classroom training in occupational skills and basic education 
are somewhat inconsistent with SDA daU on service receipt shown in Chq)ter 3» and they may in fact understate 
the incidence of these services. An analysis of these inconsistencies will be presented in our forthcoming final 
report. 
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members in the classroom training subgroup actually received classroom training. Second, 
a number of control group members in this subgroup received classroom training from non- 
JTPA providers. 

These treatment-control group dififercntials in service receipt were largely a consequence 
of the differences in the likelihood of receiving each service, reported in the first three 
columns, and not primarily a consequence of a diflfercnce in the amount of service per service 
recipient To sec this, note that among service recipients only, the average hours of receipt 
of classroom training in occupational skills were similar fortheueatmentgroup.at 722^ 
and the control group, at 843 hours." Likewise, among recipients of basic education, the 
hours of service receipt wert also similar: 348 hours for the treatment group, and 400 hours 
for the control group. It is, of course, possible that the quality of the instruction received from 
non-JTPA providers was different from that received from JTPA. 

Finally, note that 17.1 percent of the treatment group in this service strategy subgroup 
received job search assistance, and 1 1.3 percent received miscellaneous services, the last two 
categories in the panel. 

The OJT/JSA Subgroup, In the OJT/JSA subgroup 28.5 percent (second panel of 
Exhibit 4,8) of the treatment group received on-the-job training and 26.5 percent received job 
search assistance. Again, the main reason why some women did not receive one of these two 
services was that 44.6 percent of the treatment group in this service strategy never enrolled 
in JTPA during the follow-up period (Exhibit 3.12 in Ch^tcr 3). 

As expected, the likelihood of receiving on-the-job training was higher for -women in the 
OJT/JSA subgroup than for those in the other two service strata subgroups. The likelihood 
of receiving job search assistance was also higher for the OJT/JSA subgroup than for the 
classroom training subgroup, although it was only slightly higher than that for the other 
services subgroup. 

Regarding treatment-conUol group diflferences, recall that we do not have data on 
receipt of on-the-job training from non-JTPA providers. But since few of those providers 
offer OJT, the treatment-control group differences we report from JTPA providers 
are probably reasonably reliable indicators. If so, these estimates suggest that in the 
OJT/JSA subgroup treatment group members were more likely than control group 
members to receive on-the-job training (28.5 percent versus 0.7 percent). Column 6 
indicates that JTPA provided an additiOTal 104 hours of OJT, averaging oyer all treatment 
group members. This relatively small trcatment<ontrol group differential in the amount of 
OJT received per assignee reflects the fact that not all assignees in the OJT/JSA treatment 
group received OJT. 

18. Again» these numbers are derived by dividing the tvcragc number of hours of service receipt per 
treatment or control group member (column 4 or 5) by the corresponding proportion receiving the service 
(column 1 or 2, divided by 100). 
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Among service recipients only, the average number of hours of OJT receipt was 429 (3 
divided by 0.7 percent) for the control group and 375 (107 divided by 28.5 percent) for the 
treatment group. Thus, similar to the case for adult women in the classroon' training 
subgroup, the positive treatment-control group difference in hours of OJT service shown in 
column 6 was almost exclusively the consequence of the dififerencc in the likelihood of 
receiving OJT, and not of a dififerencc in the amount of the service received by those who 
actually did receive it. 

The treatment-control group differences in the receipt of classroom training and basic 
education were small in the OJT/JSA subgroup. Recall that data limitations precluded our 
calculating estimates of differences in the receipt of the last two cat^orics, job search 
assistance and miscellaneous services. 

The Other Services Subgroup. Because of the same data limitations we could not 
produce treatment-control group comparisons for these last two cat^orics, the two main 
types of services received by adults in this subgroup. We do know, however, that 62.4 percent 
of the treatment group in Ae other services subgroup of adult women ultimately enrolled in 
JTPA (Exhibit 3.12 in Chapter 3) and that 82.3 percent of those vAxo enrolled received job 
search assistance, miscellaneous services, or both (Exhibit 3. 19). 

The last panel in Exhibit 4.8 also indicates that although few members of the treatment 
group in the other services subgroup received on-the-job training or woric experience, 19.1 
percent received classroom training in occupational skills, and 10.7 percent received basic 
education. In the four catpgories of services for which we can calculate the difference in 
service receipt between the treatment group and the control group, the differences were small 
in absolute terms . The service differential in the case of basic education is a point that we will 
address further in the section on high school attainment below. 

Summary of Diffrtrenccs in Service Receipt The preceding comparisons of service 
receipt across service strategy subgroups and between thetreatmentand control groups within 
each strategy can be simimarized as follows: 

• Many women in the treatment group did not receive the primary service 
recommended for them, either because they were never enrolled in JTPA or 
because they received a service other than the primary one recommended for 
them. 

• Nevertheless, the three service strategy subgroups, which were formed on the 
basis of service recommendations by JTPA intake staflf, do appear to represent 
distinctly different clusters of services actually received and are therefore useful 
for analyzing the e ffects of alternative service strategies. 
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• Within each service strategy subgroup most women who enrolled in JTPA 
received <»e or both of two key services, which varied by service strategj^: In 
the classroom training subgroup these were classroom training in occupational 
skills and basic education; in the OJT/JSA subgroup they were on-the-job 
training and job search assistance; and in the other services subgroup they were 
job search assistance and miscellaneous services . In all three subgroups the main 
reason why a portion of the treatment group did not receive one of the two key 
services in the strategy was £adlure to enroll in JTPA. 

• The services actually received by each service strategy subgroup are consistent 
with the rcconunendations of intake staff to a considerable extent, but the>' 
differed in some important ways. Most important was the finding that only 28 .9 
percent of adult female treatment group members reconmiended for OJT (the 
OJT/JSA service strata) actually received that service. Because 27.0 percent 
of adult female treatment group members recommended for OJT received job 
search assistance, it is most j^propriate to characterize the service strategy as 
one based on placement in employment, with or without subsidized training. 

• Treatr.ient-control group differences in the average amount of service received 
were relatively modest, \rfiere those differentials could be measured. The 
average amount of additional occupational skills training received by treatment 
group members in the classroom training subgroup was only 110 hours. 
Similarly, the treatment-control group difference in receipt of OJT by women in 
the OJT/JSA subgroup was only 104 hours. These modest service differentials 
reflect the feet that not all treatment group members received these services and, 
in the case of occupational skills training, that some control group members 
received the service. 

Because of the marked difference in the nature of the services received across the three 
subgroups, it is importii^-t to recognize that the costs of the three service strategies are also 
likely to have been quite different. Thus, the differences in impacts we report later in this 
section may not correspond to differences in cost-effectiveness. Our forthcoming final report 
will present a benefit-cost analysis that takes these differences in costs into account. 

Impacts on High School Attainment As part of the training described in this section, 
some of the women in the sample achieved a high school diplana or General Educational 
Dcvelq)ment (GED) certificate. Exhibit 4.9 shows the impzcX of JTPA on the attainment of 
a high school credential by women who reported participating in school or training at some 
time during the follow-up period. We term such credentials "training related.** Treatment and 
control group attainment rates are shown for adult women overall and for high school 
dropouts only within each service strategy subgroup. The treatment-control group differ- 
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Exhibit 4.9 Impaas on Anainmem of a Training-Relaxed High School 

Diploma or GEO Certificate: JTPA Assignees Overall and High 
School Dropout Subgroup, by Service Strategy Subgroup 







Percentage attaining a 
training-related high school credential'' 


size* 

(1) 


Control 

Assignees group 
(2) (3) 


Difference, 
in % pts. 
(4) 






Qassroom training subgroup 




Full sample 


2,390 


7.3% 2.9% 


4.4%*** 


High school dropouts 


605 


29.2 11.3 


17.9*** 






OJT/JSA subgroup 




Full sample 


1,955 


2.6 3.5 


-0.9 


High school dropouts 


574 


9.1 10.9 


-1.8 






Other services subgroup 




Full sample 


1,064 


5.3 3.3 


2.1** 


High school dropouts 


336 


17.4 9.8 


7.6** 






All subgroups 




Full sample 


5,409 


5.2 3.2 


2.0*** 


High school dropouts 


1,515 


19.1 10.8 


8.2*** 



Sources: Unadjufted frequeocies baaed on Background Informatioo Fonn reaponaes and Firat Follow-up 
reaponaec. 

a. Treatment and control groups combined. 

b, 'Attainmeot of a traiomg-rdated high acbool credential* U deHoed as tbe combmatioo of having 
received aome achool or training aervice and having attained a bi^ acbool diploma or General 
Education Development ceniftcate at aome time during tbe 18*moath follow-up period. 

* StatiAically significant at the . 10 level, ** at the .05 level, *** at tbe .01 level (two-tailed tett). 

ences shown in column 4 measure tbe additional educational attainment attributable to having 
access to JTPA. 

As shown in the last panel of the exhibit, only S.2 percent of the treatment group overall 
and 3.2 percent of the control group overall reported receiving a training-related high school 
credential at some ttue during the follow-up period. Thus» even though the treatment-control 
difference is statistically significant, the impact of the program on educational attainment is 
quite small when measured as a prc^rtion of all women. This is because 71.8 percent of the 
treatment group already had a high school diploma (see Exhibit 3.13 in Chapter 3). 

Among high school dropouts only (those women without a high school diploma or GED 
certificate upon application), the impact of the program was more substantial. As shown in 
the last line of the exhibit, 19. 1 percent of the high school dropouts in the treatment group 
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gained a training-related high school credential during the follow-up period, whereas 
only 10.8 percent of the dropouts in the control group did. We can therefore conclude 
that 8.2 percent of the dropouts assigned to the treatment group received a high school 
diploma or GED certificate as a result of their access to JTPA.^' 

As might be expected, this eflfect was concentrated in the classroom training subgroup, 
in which women were the most likely to receive basic education and in which the treatment- 
control group difference in the likelihood of receiving basic education was the largest. In this 
subgroup 29.2 percent of high school dropouts in the treatment group received a training- 
related high school credential, whereas only 1 1 .3 percent of the corresponding control group 
did. There was also a significant effect on educational attainment in the other services 
subgroup, which as we saw earlier, exhibited a treatment-control group difference in the 
likelihood of receiving basic education nearly as large as that in the classroom training 
subgroup. There was no effect on training-related high school attainment in the OJT/JSA 
subgroup, in which there was almost no treatment-control group difference in the receipt of 
basic education. 



Differences in Baseline Characteristics across Service 
Strategy Subgroups: JTPA Assignees 

As explained more fully in Chapter 2, JTPA staff recommended sample members for the three 
service strategies on the basis of individuals* work experience and educational needs as well 
as their personal preferences. Variations in impacts among the three service strategy 
subgroups will therefore reflect not only the differences in services received by these groups, 
but also differences in their personal characteristics. 

Exhibit 4.10 shows selected baseline characteristics of all aduh female assignees and 
those in each of the service strategy subgroups.^ Although most of the differences in 
characteristics among the three subgroups were not large, it does appear that program staff 
tended to recommend the OJT/JSA service strategy for the most job-ready applicants. 
Assignees in the OJT/JSA subgroup were less likely than those in the other two subgroups, 
especially the classroom training subgroup, to be fistcing the barriers to employment 
represented by welfare receipt or limited recent work experience. They were also slightly 
older, on average, than assignees in the classroom training subgroup and were more apt to 



"19. The difference is 8.2 percent because of rounding. 

20. For t more detailed description of the baseline characteristics of these subgroups, see Bloom (1991 ). 
Note, however, that the data in Bloom (1991) cover all JTPA ^plicants randomly assigned to treatment or 
control statiis» >vhereas Exhibit 4.10 includes only the adult female treatment group in the smaller 18*month 
study sample. Appendix A in the present report compares the treatment and control groups in this sample. 
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Exhibis 4. JO Selected Baseline Characteristics: JTPA Assignees, by Service Strategy Subgroup 







Classroom 


OJT/ 


Other 




All 


traiitinff 


JSA 


senices 




subgroups 


subgroup 


subgroup 


subgroup 


Characteristic 


(1) 


a) 


(3) 


(4) 










52.1% 


White* non-HitpAnic 


54.5% 


49.2% 


62.6% 


BUck, non-Hiipamc 


30.8 


33.8 


27.5 


30.1 


HispAnic 


11.4 


13.0 


7.6 


14.1 


American Indian or AUibui Native 


1.0 


1.2 


0.9 


1.0 


Aiian or Pacific Iilander 


2.3 


2.8 


1.4 


2.8 


Barriers to employinait 










Receiving cash welfare^ 


37.9% 


47.2% 


29.7% 


32.3% 


No high achool diploma or 








31.9 


GED certificate 


28.2 


26.2 


28.6 


wonceo teat man i ^ wocu 










in paat 12 montha 




57.4 


49.1 


53.8 


Number of bamen 










pione oi uic vTQYc 


28.6 


24.5 


33.1 


29.9 


One of the above 






^U. I 


34.9 




11.% 


31.6 


23.9 


25.9 




9.2 


11.0 


6.9 


9.2 


Work and training histories 










Ever employed 






on s% 


85.2% 


Mean individual earmngt 










in pait 12 morthi 


$2,362 


$2,092 


$2,665 


$2,441 


Houriy earnings in moft recent job 










Never employed 


14.4% 


18.1% 


9.5% 


14.8% 


Leatthan$4 


34.3 


32.5 


38.5 


30.8 


$4 or more 


51.3 


49.4 


52.0 


54.4 


En^k>yed upon application 


14.6 


14.3 


15.5 


13.9 


Pieviouily received occupational 










training 


46.1 


46.6 


45.4 


46.3 


Public assistance status 










Receiving any public aiaistance^ 


58.7% 


64.6% 


54.3% 


53.6% 


Receiving AFDC 


33.8 


43.6 


25.2 


27.4 


Receiving food Aampt 


50.3 


56.0 


46.2 


45.2 


Receiving other public aaiiAance ^ 


18.4 


19.9 


16.4 


19.0 



1^2 
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Exhibit 4. 10 Selected Baseline Characterisiics: JTPA Assignees, by Service Straiegy Subgroup 
(continued) 



Characuristic 



All 
subgroups 

0) 



Classroom 
training 
subgroup 

(2) 



oni 

JSA 
subgroup 

(3) 



Other 
services 
subgroup 
(4) 



AFDC histoiy 

Never AFDC cmc bead 
AFDC c*«c head less thtn 2 yc*r£ 
AFDC c»»c bead 2 yc«ri or more 
JTPA required for welfare, food 

stamps, or WIN program ^ 
Household composition 

No ipoute or own child present 
Own child under age 4, 

no ipouse present 
Own child, none under 4, 

no ipoute present 
Spouse present, with or 
without own child 
Family income in past 12 months 
< $3,000 
$3,000 - $6,000 
$6,001 - $9,000 

> $9,000 

Living in public housing 
Yes 
No 

Age at random assignment 
22-29 
30-44 
45-54 

> 54 
Mean 

Sample size 



23.1 
27.9 



41.6% 

24.8 

33.5 



55.5% 

21.7 

22.8 



53.9% 

21.7 

24.4 



12.3% 


12.2% 


12.0% 






15.4% 


25.4% 


28.3% 


20.1 


22.7 


17.6 


16.1 


34.9 


40.4 


34.5 


33.2 


22.6 


21.5 


22.5 


22.3 


30.8% 


28.5% 


33.3% 


31.4% 


33.7 


36.9 


29.5 


34.3 


16.1 


16.7 


16.0 


15.1 


19.4 


18.0 


21.3 


19.2 


11.8% 


14.0% 


10.6% 


9.6% 


88.2 


86.0 


89.4 


90.4 


44.2% 


47.0% 


43.4% 


39.7% 


43.1 


44.6 


41.4 


42.7 


8.0 


6.1 


10.5 


8.0 


4.7 


2.3 


4.7 


9.6 


3B.3 


32.0 


33.8 


35.2 


4,465 


2,966 


1.562 


937 



«. AFIX:. Gei>aiJAMbi«»ot,orolhwweljW«eMq«foodiuinpt. 

b. 'Any public SMiiUwoe' iocluka the foUowing •ourow of •••irtmce: 
botaiof MtiiUDce. aid other ci»h aniitsooc. 

c. 'Otfaer public MiisUnoe' iocludai uoanfrfoymeai imumice* boutiac 

d. WIN M Ihc federal Woifc b>ca*ive progimia 



AFDC. food tuunpt. iBcxnployfneM 
•wisUDoe, snl other (aoo-AFDC) oMh Msi 
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have worked before, and to have had higher earnings in the year preceding their application, 
than those in the other two subgroups. 

Assignees in the other services subgroup were older, on average, than those in the other 
two subgroups, ITiey were also slightly less well educated, but (perhaps because of their age) 
they were the least likely to have received a low wage (less than $4.00 hourly) on their most 
recent job. In other respects, the other services subgroup tended cither to be intermediate 
between the classroom training and OJT/JSA subgroups (for example, in terms of ethnic mix, 
earnings in the past year, and family income) or to resemble the OJT/JSA subgroup (for 
example, in terms of household composition and public assistance status). 

Because of these differences among the women in each of the three service strateg\' 
subgroups, one must be careful in comparing program impacts across the three groups. The 
impacts presented in the next subsections reflect the effects of the program on the kinds of 
people recommended for each subgroup. If the same service strategy were recommended for 
a different set of people, there is no guarantee that the same impacts would be obtained. Thus, 
although the analysis can identify a strategy (or strategies) that was working, or not working, 
for the group of people recommended for that strategy, we cannot tell v^ether the labor market 
outcomes of one subgroup could be improved by substituting a different set of services. 



Monthly Earnings Trends: Treatment and Control Groups 
WITHIN Each Service Strategy Subgroup 

As shown in Exhibit 4.11, the monthly earnings profiles of the treatment and control groups 
were markedly different across the three service strateg>' subgroups, both in the level of 
control group earnings and in the contrast between treatm.ent group earnings and control group 
earnings. ^ 

In fact, the trends for the control groups in the three service strategies bear out our 
conclusion regarding the baseline characteristics of the three subgroups — that program staff 
assigned the more employable applicants to the OJT/JSA service strategy. Earnings in the 
absence of JTPA services, as measured by the control group level, were much higher for the 
OJT/JSA subgroup than for the classroom training control group in the first quarter after 
random assignment, and then continued to be higher over the entire follow-up period. Control 
group earnings in the other services subgroup were intermediate between these two sub- 
groups. 

In all three subgroups, earnings in the absence of JTPA services (the control group level) 
showed a pronounced upward trend throughout the follow-up period. Treatment group 
earnings were at or abo\'e the corresponding control group level throughout the follow-up 
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Exhibit 4. 1 1 Average Monthly Earnings of Adult Women: Treatment Group and Control Group, by Service Strategy Subgroup 



Classroom Training Subgroup 



OJT/JSA Subgroup 



Other Services Subgrou] 



$1,000 



$800 



8 $600 
>^ 

a 

o 

a 

o $400 

C 
> 

< 



$200 



$0 




. , . , ■ . . , i . , \ 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 » » I 1 1 1 1 i « » • L_ 

3 6 9 12 15 18 0 3 6 9 12 15 18 0 ? 6 9 



Months ifter random assignment 
—Treatment Control 



Sources: Estimates based on First Follow-up Survey responses and earnings data from state UI agencies. 

Notes: Sample size, classroom training subgroup: treatment group « 1.916. control group « 931 : OJT/JSA subgroup: treatment group « 1.538. control group = 7 
treatment group = 922, control group s 418. Estimates are regression-adjusted to control for differences in baseline characteristics between the treatment group 
Appendix D. 
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period for two of the three subgroups— OJT/JSA and other services— and in the post- 
program period (months 10 through 18) for all three groups. 

Impacts on Earnings: JTPA Assignees and Enrollees in Each 
Service Strategy Subgroup 

The patterns of quarterly impacts on earnings shown numerically in Exhibit 4, 12 are highly 
consistent with what one would expect for each of the three service strategies. The classroom 
training strategy involves an initial investment of time in classroom training that can be 
expected to delay participants' employment. Once trained, however, participants should 
command higher earnings than they would have without training. Moreover, to the extent 
that classroom training improves trainees' human capital, th^e gains should persist for a 
relatively long time. 

As shown in the top panel of the exhibit, this is exactly the pattern of Title II-A impacts 
estimated for the classroom training subgroup. The estimated impact on earnings grew 
continuously overthe foUow-up period, from a statistically significant loss of $-70 in the first 
quarter to statistically significant gains of S144 and SI 88 in the fifth and sixth quarters. 

In contrast, the OJT/JSA strategy should involve no delay in cnq)loyment. If anything, 
JTPA assignees recommended for this service strategy should show earnings gains early in 
the follow-up period, because of quicker job placements than they would have experienced 
in the absence of the program. And if participants in on-the-job training positions develop 
their human capital or learn job search sldlls that shorten any subsequent periods of 
unemployment, the early earnings gains should persist over the longer term. 

This is in feet the pattern of effects shown in the middle panel of E:dubit 4.12, Assignees 
in the OJT/JSA subgroup experienced relatively imiform earnings gains throughout the 18- 
month follow-up period, with statistically significant gains in the range of $109 to $144 in 
all but the second quarter after random assignment.^^ 

Finally, the other services strategy can be expected to have relatively rs^id effects, 
because it comprises a variety of less intensive services, many of which are directed toward 
immediate employment. The e3q)ected duration of these effects is less clear, however, than 
it is for the other two subgroups. On the one hand, because many of these services involve 
little human capital development, one might expect these effects to be relatively short-lived. 



21. As noted earlier, the earnings estimates include wages paid to JTPA participants in on-the-job 
training positions. The program reimbursed employers about $650 per adult female OJT participant 
during the 1 8-month follow-up period. Within the OJT/JSA subgroup, the reimbursement was about $ 1 50 per 
adult female assignee over the follow-up period. 
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Exhibit 4. 12 Impaas on Earnings: JTPA Assignees and Enrollees, 
by Service Sir egy Subgroup 







Impact per assi/mee 


Inferred impact per 




Control 








mean 


In$ 


As%of(l) 


enrollee. in $ 


Period 


(1) 


a) 


0) 








Classroom training subgroup 




Quarter 1 


S 714 


$ -70* 


-9.8% 


•100 


2 


938 


S 


0.5 


7 


3 


1,066 


52 


4.9 


74 


4 


1,189 


79 


6.6 


112 


5 


1,253 


144** 


11.5 


205 


o 




188*** 


15.3 


268 


All quarters 


6,391 


398 


6.2 








OJT/JSA subgroup 




Quarter 1 


$ 1,143 


$ 144*** 


12.6% 


273 


2 


1,379 


81 


5.9 


153 


3 


1,449 


129** 


8.9 


245 


4 


1,520 


109* 


7.2 


207 


5 


1,546 


142** 


9.2 




6 


1,570 


138** 


8.8 


263 


All quarters 


8,607 


742** 


8.6 


1,409 






Oth^r services subgroup 




Quarter 1 


$ 960 


$ 39 


4.1% 


67 


2 


1,198 


132 


11.0 


223 


3 


1,248 


220** 


17.6 


372 


4 


1,471 


22 


1.5 


37 


5 


1,535 


2 


0.1 


3 


6 


1,548 


42 


2.7 


71 


All quarters 


7,960 


457 


5.7 


773 



Sources: EttimAtes bued on Fine Follow-up Surv^ raponM and emingt dau from fUiJt UI agencies. 
Nous: Saii^le size, dusroom training subgroup: assignees « 1,916, ooouol group « 931; OJT/JSA 
subgroup: assignees » 1,538, control group « 749, other services subgro^): assignees » 922, control gro^p 

418. Estimates are regression-adjuxed to control for differences in basdtne characteristics bccweeo the 
trealmeot group and control group; aee Appendix D. 

* Statistically significant at the . 10 level, ** at the .05 Icvd, *** at the .01 level (two-tailed test). 
Significance levels for column 3 are identical to those in column 2. Tc«U of statittical aignificanoe were 
not performed for column 4. 
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But several previous studies of low-intensity emplo>'ment and training services for women 
receiving welfieire indicated earnings gains that did not noticeably decline over the first 18 
months or more (Gueron and Pauly, 1991). Most of the programs studied, however, were 
mandatory for AFDC applicants, recipients, or both and did not include other adult women. 
Thc>' therefore may not be comparable to JTPA, which serves a population of predominately 
voluntary participants, most of whom are not wel&re recipients. 

The third panel of Exhibit 4.12 indicates the pattern of JTPA impacts on earnings for the 
other services subgroup was one of relatively quick, but short-lived, effects. Estimated 
e;^T"^"g$ gains sharply declined in the post-program period, from a statistically sig nifi ca n t 
intact of $220 in tiie third quarter to a statistically insignificant gain of $42 in the sixth 
quarter. 

Thus, the patterns of impacts on earnings over time suggest that simply comparing the 
effects on total earnings over the 18-month follow-up period for the three subgroups may be 
somewhat misleading. For the follow-up period as a whole, the other services subgroup had 
an average earnings gain that was s(Mne\diat larger than that of the classroom training 
subgroup ($457 versus $398). But the eflfect on the other sci^iccs subgroup peaked midway 
into the follow-up period and had begun to disappear by the end, whereas the effect on the 
classroom training subgroup — ^and the OJT/JSA subgroup — was still large and significant in 
the last two quarters of the period. 

If these patterns persisted beyond the 18-month follow-up period, the classroom training 
and OJT/JSA subgroups might witness substantially larger l<Hig<^rm effects than the other 
services subgroup. More speculatively, if the impacts of the classroom training strategy 
continued to grow over the longer term, wiiile the impacts of the OJT/JSA strategy remained 
stable or declined, the classroom training strategy could well have the largest long^rm 
effects. These issues will be addressed for a longer follow-up period in our forthcoming fiyoal 
report. 

It is important to remember, however, that what matters fi-om a policy perspective is not 
just the absolute size of the impacts on the service strategy subgroups, but the impacts relative 
to the costs of each strategy. Again, the final report will also present a benefit-cost analysis 
that compares the impacts of the three JTPA service strategies and their costs, to determine 
which one or ones were cost-effective. 

Even then, however, we will only be able to say which sendee strategies were cost- 
effective for the individuals recommended for each one. There is no guarantee that a service 
strategy that has been shown to be cost-effective for one group will be cost-effective for a 
different group. 
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The fourth column of Exhibit 4.12 shows the imparts on earnings of the typical JTPA 
enrollee that can be inferred from the impacts per assignee in column 2 of the exhibit, if we 
assume that the program had no effect on Ueatment group members who did not enroll m 
JTPA For the classroom training subgroup these impacts per JTPA enrollee were uniformly 
37 percent larger than the impacts per assignee; for the OJT/JSA subgroup they were 80 
percent larger; and for the other services subgroup, 60 percent larger. (The adjustment to the 
impact per assignee varies among subgroups because the JTPA enrollment rate, upon which 
the adjustment is based, varied by subgroup.) 

It is important to bear inmindthattheinferredimpacts per enrollee showninExhibit4.12 
probably overstate somewhat the true impacts on JTPA cnroUecs. These estimates are based 
on the assumption that there was no impact on treatment group nonenroUees. As noted m 
Chapter3,somenonenrollees received some JTPAscrvice,however minimal that service, and 

therefore their earnings may have been afferted by the program. 

Impacts on Employment: JTPA Assignees in Each Service 
Strategy Subgroup 

As we did for adult women overall, we now examine the JTPA effects on employment for 
women in each of the three service strategy subgroups. Exhibit 4. 13 displays in columns the 
estimated program impacts per assignee on ihc percentage employed, the average number 
of weeks worked, and the average number of hours worked, with results for the three 
subgroups again displayed in panels. 

Not surprisingly, the patterns of estimated effects over time for employment roughly 
parallel the patterns for earnings shown in the previous subsection. However, as shown in the 
top panel of the exhibit, the only statistically significant impacts on the employment of 
women in the classroom training subgroup were reductions in average weeks and hours of 
work of about half a week in the first quarter and an increase in average weeks worked of about 
half a week in the sixth quarter. 

In the OJT/JSA subgroup, the percentage employed increased significantly in the fifth 
quarter only, and there were no significant effects on weeks worked. Average hours worked 
significantly increased in the first, third, and fifth quarters, however; and for the follow-up 
period as a whole average hours worked were nearly 100 hours higher than they would have 
been in the absence of the program. 

Adult female assignees in the other services subgroup had significantly higher employ- 
ment rates, average weeks vvorksd, and average hours worked than the corresponding control 
group in the third quarter, where we earlier saw significant earnings gains. They also had 
significantly higher employment rates in the second and fifth quarters. For the follow-up 
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Exhibit 4.13 Impacts on the Percentage Employed and on the Mean Number of "Weeks 
and Hours Worked: JTPA Assignees, by Service Strategy Subgroup 





Percentage emplcyed 


Weeks woHced 


Hours worked 




Conxrol 


Impact, 


Control 


Impact, 


Control 


Impact, 




mean 


in % pts. 


mean 


in weeks 


wean 


in ncfurs 


Period 


(1) 


a) 


(3) 


(4) 


PJ 










Classroom training subgroup 






Quftiter i 


39.6% 


-2.1% 


4.1 


■OA* 


147 


-24* •* 


2 


46.7 


-1.0 


5.1 


-0.2 




-14 


3 


49.7 


1.0 


5.5 


0.0 


203 


-6 


4 


54.5 


0.4 


6.0 


0.0 


220 


-1 


s 


54.4 


2.5 


6.4 


0.1 


230 


3 


6 


55.5 


2.5 


6.1 


0.4* 


220 


15 


All quarters ^ 






33.7 


-0.1 


1,222 


-30 








OJT/JSA subgroup 






Quflrter 1 


60.1% 


1.3% 


6.2 


0.3 


234 


23** 


2 


62.3 


1.7 


7.0 


0.1 


268 


11 


3 


63.7 


1.9 


7.1 


0.2 


273 


21* 


4 


65.7 


2.2 


7.6 


0.0 


288 


13 


5 


66.0 


3.4* 


7.8 


0.2 


290 


19* 


6 


67.4 


1.7 


7.8 


0.0 


290 


15 


All quarters ^ 


82.8 


2.4 


43.8 


0.7 


1,655 


97* 








Other services subgroup 






Quarter 1 


47.0% 


3.0% 


4.8 


0.5 


172 


7 


2 


S2.3 


6.5** 


5.8 


0.5 


213 


7 


3 


S4.9 


6.4** 


6.1 


l.O*" 


224 


33.. 


4 


60.5 


-0.6 


6.6 


0.5 


252 


10 


5 


58.5 


4.5* 


6.9 


0.7** 


261 


9 


6 


61.6 


2.3 


7.1 


0.5 


267 


13 


All quarters ^ 


74.8 


4.3* 


37.2 


3.7** 


1,385 


79 



Sources: Ettimatei bued on Pint FoUowHip Surv^ iMpontet tod earningt d«U from lUte UI «fenciet. 
Notea: Sui^le tize, cliMroom Utining mbgroup: tMifneet > 1,916, control f roup - 931; OJT/JSA tubfroup: 
Attignm * 1,538, control group * 749; other tervioet lubfroup: Mtigneee > 922, control group « 418. 
EctimatM tre r^greMioo-adjutted to control for difTerenoe* in bwdine chtracteriitict betM«en the tieAtment 
group and control group; iee Appendix D. 

a. For columnc 1 tnd 2 Cpetcentitc eonployed") thit row sbo^t the perccnt^e of control group member* who 
rtporud being cn^loyed at any time during (he foliow-up period and the aatimaiad impact on thit percentage 
for aaaigneaa, reipectively. 

• Sutittically tignificant at the .10 level. at the .05 level, at the .01 level (two-taUed Icat). 
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period as a whole this subgroup was significantly more likely to be employed at some time and. 
on average, worked 3.7 more weeks than they would have without access to JTPA. 

As noted earlier, because these three employment outcomes are averages over all 
assignees in each subgroup, the estimated effects on each are not independent. In particular, 
any impacts on average weeks and hours employed for all assignees may simply be a reflection 
of a higher proportion of women working. The next subsection focuses on the average weeks 
worked and hours worked per week of only those assignees who worked, to distinguish the 
added hours and weeks of work generated by an effect on the employment rate alone and those 
generated by effects on the hours and weeks worked by those who were employed. 

Impacts on the Components of Earnings: JTPA Assignees 
IN Each Service Strategy Subgroup 

As shown earlier in this chapter, the average earnings of JTPA assignees can be decomposed 
into four multiplicative parts: the proportion of all assignees who were employed, the average 
number of weeks worked per worker, the average number of hours worked per week worked, 
and the average earnings per hour worked. The estimated impacts on these components of 
earnings tell us whether access to JTPA made it more likely that women would be employed 
at all and whether those who were employed were more likely to work more steadily, to have 
a full-time job, or to earn more per hour. Exhibit 4.14 presents these estimates as a percentage 
of the corresponding control group mean for each service strategy subgroup. 

Recall that because the percentage impacts on the four components in each row of the 
exhibit sum to approximately equal the percentage impact on earnings in the same row, the> 
indicate the relative contribution of each component to that impact on earnings. The exhibit 
shows that the composition of the earnings gains differed substantially across the three service 
strateg\' groups. 

In the classroom training subgroup (top panel) the 6.2 percent impact on earnings over 
the entire follow-up period was predominately the result of an 8 .9 percent impact on earnings 
per hour worked, which offset a small decrease in the duration of employment spells (as 
measured by average weeks worked per worker) and a small decrease in full-time work (as 
measured by average hours worked per week worked). For this subgroup the program had 



22. Sec Appendix D for a description of the method by which the estimates were calculated 

23. Again, we did not calculate statistical significance levels for the estimates in the earnings componcnis 
exhibits, because the estimates in the last three columns were calculated indirectly. 
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Exhibit 4.14 Percentage Impacts on Earnings and Its Components: JTPA Assignees,, 
by Service Strategy Subgroup 



Workers Weeks Hours worked Earnings 

Earnings per worked per week per hour 

per assignee assignee per worker worked worked 

Period (1) (2) 0) W (IL^ 



Classroom training subgroup 



Quarter 1 


-9.8% 


-5.3% 


^.9% 


-1.3% 


8.05^ 


2 


0.6 


-2.1 


-0.9 


-4,8 


9.0 


3 


4.9 


2.0 


-2.5 


-2.4 


8.0 


4 


6.6 


0.7 


0.1 


-1.1 


7.0 


5 


11.6 


4.6 


-2.7" 


-0.7 


10.4 


6 


13.3 


4.5 


2.5 


-0.1 


7.8 


All quarters 


6.2 


1.8 




-9 1 


8.9 








OJT/JSA subgroup 






Quarter 1 


12.6% 


2.1% 


2.5% 


4.8% 


2.6% 


2 


5.8 


2.7 


-1.9 


3.5 


1.5 


3 


8.9 


3.1 


0.2 


4.1 


1.3 


4 


7.2 


3.3 


-2.9 


4,1 


2.6 


5 


9.2 


5.2 


-2.5 


3.9 


2.4 


6 


8.8 


2.5 


-1.9 


4,6 


3.4 


All quarters 


8.6 


2.9 


-1.3 


4.2 


2.6 








Other services subgroup 






Quarter 1 


4.1% 


6.4% 


3.1% 


-4.9% 


-0.2% 


2 


11.0 


12.4 


-3.9 


-4,2 


7.3 


3 


17.6 


11.7 


3.6 


-0,9 


2.5 


4 


1.5 


.1.0 


9.1 


-3.6 


-2.5 


5 


O.I 


7.6 


2.4 


-6,2 


-3.2 


6 


2.7 


3.8 


3.7 


-2.6 


-1.9 


All quarters 


5.7 


5.8 


3.9 


-3.7 


0.0 



Source*: E*limiie» bttcd on Finl Follow-up Survey rc«ponie» and ctmingt data from suie UI agencies. 
Note*: Sample aizc. clasiroom iraining aubgroup: aiaignoc* « 1.916. control group « 931; OHVJSA subgroup: 
assignee* « 1,538. control group « 749; other services subgroup: assignees « 922. control group = 418. Est 
are regression-adjusted to control for differences in baseline characlcristics between ihe treatment group and 
control group; see Appendix D. Columns 2 through 5 display the impact as a percentage of the corresponding 
control mean (not shown). For colunm 2 this means the impscl on the employment rate U calculated as a 
oercenuge of the mean rate for the control group. Tests of statistical significance were not performed for any 
of the columns in this exhibit. 
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only a small positive effect on the percentage employed.^' These results are consistent with the 
view that classroom training involves an initial investment of forgone emplov-ment and 
earnings that should then lead to higher rates of pay. 

n,e 8 6 percent impact on the earnings of those in the OJT/JSA subgroup (which was 
statistically significant in Exhibit 4.12) was more evenly distributed across the components 
of earnings. TTiroughout the follow-up period this subgroup experienced mcrcases m 
employment rates, hours worked per week worked, and earnings per hour worked. There was 
little effect however, on the stability of employment-weeks worked per worker. Overall this 
pattern is consistent with a program strategy of placing participants as soon as poss.b e m 
either OJT slots or regular jobs, which should lead to higher employment rates more ftili-t.me 
work, and work at higher wage rates than the jobs the treatment group would have found m 
the absence of the program. 

The 5 7 percent impact on earning? the other services subgroup was apparently due 
entirely to a 5.8 percent increase in this group's employment rate. The program impact on 
weeksworked per worker(3.9perccnt)was almost exactlyoffsetbyitseffectonhoursNvorked 
per weekofemployment(0.7).Earningsperhouremployed were notaffectedbytheprogram. 

Effects on the proportion employed peaked sharply in the second and third quarters of the 
follow-up period and then diminished. 

This pattern of effects is consistent with a strategy of relatively short-term, low-intensity 
services aimed at helping individuals find jobs as quickly as possible. The strateg>' seemed 
to work in enabling assignees to find jobs sooner than they would have, on average, without 
accesstotheprogram. But itdoesnotappearthatthejobsthey found weremuch better interms 
of hours per week or rates of pay than the jobs they would otherwise have obtained. Thus, 
although assignees in the other services subgroup enjoyed substantial earnings gams over the 
first three quarters as a result of their landing jobs sooner than the control group, by the end 
of the follow-up period those gains had substantially declined. 

The results in Exhibit 4 . 14 tell a story that is seemingly very cohtiCnt and consistent with 
the goals of each service strategy. Caution must be exercised in their interpretation, however. 
As noted earlier, the impacts on weeks worked when employed, hours worked per week 
employed, and the hourly earnings of workers may reflect effects on the composit ion of the 
subgroup of assignees who worked, in addition to or instead of effects on the weeks, hours, 
and hourly earnings of specific individuals. For this reason the estimated impacts on average 
earnings per hour worked cannot be interpreted as measures of effects on productivitj'. 



24 Note thai ihe employmenl rale in this Exhibil 4.14 (column 2) is expressed as a percenlage ol ihe 
corTespondmgconuolgroupmean.Intheprecedingexhibil.4.13(colunm2).theemploymenlralc>sexpresscd 

as a pcrceniage point difference fiom the corresponding conuol group mean. 
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The analysis of impacts on latent wage rates presented in Appendix G anempts to 
distinguish between these two different types of effects. None of tlie estimated impacts on 
latent wage rates by service strategy subgroup are significantly different from zero. However, 
as noted earlier, the noncxperimental estimation technique used in Appendix G ma>' not be 
reliable for re^xsons discussed in the appendix. There may therefore be no fully satisfactor>' 
way to address the compositional issue in the present study. 



Impacts on Earnings: Adult Women in Selected Key Subgroups 

Up to this point, we have presented impact estimates for all adult women and for those in each 
service strategy subgroup. To better understand the distribution of program effects, we now 
examine the impacts of JTPA Title II-A on a number of other subgroups of women of interest 
to both policymakers and program administrators: those defined in terms of their ethnicit>', 
barriers to employment, work history, welfare history, household composition, famih' 
income, and age.^ This analysis allows us to distinguish the groups for which JTPA was 
particularly effective, which in turn should help policymakers target future research into the 
factors that lead to program success. And by identifying any groups for which the program 
was not working well, the analysis should help policymakers and program administrators 
target their efforts for improvement. 

It is important to note at the outset that any comparison of program impacts across these 
"key subgroups'' must take into account the feet that effects may vary for any of a large 
number of reasons, reflecting the many dimensions in which subgroups may differ from one 
another beyond the single, selected characteristic defining them. On average, white women, 
for example, differ from black women in a variety of ways beyond ethnicity, such as in where 
they live, their education, and their work experience. 

Furthermore, some subgroups that exhibit especially large earnings impacts may have 
been concentrated in sites with particulariy effective programs. Other successful subgroups 
may have received a particularly effective mix of program services. But the ability of any one 
of these groups to benefit more from the program may also have been due to factors not 
directly related to the JTPA program, such as conditions in the local labor market or other 
personal characteristics of the subgroup members themselves. 

To distinguish which of these sets of factors was responsible for differences in program 
impacts across subgroups, we derived three different impact estimates for each subgroup 
examined, as shown in the last three column headings of Exhibit 4.15. Column 3 gives the 

25. We sclcclcd the subgroups examined in this section based on their relevance to policy discussions, 
before we calculated thccstimatcs. In other words, we did not select them on the basis of the size or significance 
of the program effects presented here. 
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estimated impact on total 1 8-month earnings for each key subgroup, whereas in column 4 this 
estimated impact was adjusted for the subgroup's distribution across sites and in column 
for the distribution of the subgroup across both sites and service strategies. Column 1 gives 
the size of the sample for each subgroup (with treatment and control groups combined); 
column 2 shows the mean earnings of the corresponding control group.^* 

It is perhaps simplest to read the exhibit as follows. First, togauge the variation in impacts 
across key subgroups, compare the findings for each in column 3 . Then, to assess the extent 
to which the subgroup differences shown stem firom differences in the distribution of each 
subgroup across sites or service strategics, scan across the last two columns . If, for example, 
the impact estimates in column 3 differ substantially between two subgroups, but those 
adjusted for the distribution of each of the two subgroups across sites do not, one can conclude 
that the difference between the column 3 estirnates was due to factors associated with the 
site— either the nature or mix of program services or other characteristics of the local 
environment. 

In abnost all cases, the two adjusted estimates for each subgroup are quite similar to the 
estimate in column 3, m terms of both size and statistical significance.^' This means that the 
differences in impacts across the subgroups probably do not reflect differences in the 
distribution of the subgroups across sites or service strategies. They may, of course, reflect 
differences in the personal attributes of the subgroups' members beyond the characteristic 
defining the subgroups. 

Note that Exhibit 4.15 reports on two tests of statistical significance. The first, indicated 
by asterisks beside the impact estimate, is the conventional t-test of significance for an 
individual impact estimate. The results of this test determine our level of confidence that the 
estimated impact differs fi-om zero. The second lest of statistical significance, indicated in the 
row following each set of subgroups, is a joint F-test of whether the estimated impacts in that 
set of subgroups differ significantly fi-om one another. 

Although a number of the impact estimates for individual subgroups are statistical 1>' 
significant, the F-tests show that in almost none of the sets of subgroups do the impact 
estimates differ significantly fi-om one another at the .10 level. This means that, in general, 
although we are confident that JTPA had a positive impact on certain subgroups, we cannot 
be sure whether it had an impact on the other subgroups, and we cannot say with confidence 
that the impact of the program was greater for the former subgroups than for the latter. The 
estimates simply are not precise enough to make such distinctions, even in cases of relativeh' 

26 The estimates in columns 4 and 5 adjust the distribution of each subgroup to equal the distribution of 
adult women overall across sites (column 4) or across sites and ser%'ice strategies (column 5) For a full 
description of the methodology for deriving these estimates, sec Appendix D. 

27. The only notable exception is the estimated impact on the earnings of Hispanic women 
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large differences in magnitude beween the estimates for any two subgroups in a set. These 
differences could be due to random sampling error. 

It is important to bear in mind that the precision of the estimates for any given subgroup 
depends on its sample size. The estimated impacts for small subgroups are subject to large 
sampling error. Thus, for small subgroups, only relatively large impacts or large differences 
in impacts relative to other subgroups can be detected >vith confidence. 

In the discussion of estimated impacts on key subgroups that follows, we focus on the 
estimates in column 3, because they are experimentally based and thus are the most direct and 
reliable estimates of the impact. The adjusted estimates in columns 4 and 5, on the other hand, 
rely on linear adjustments for differences in site and service strategy distributions and 
therefore may be subject to model specification error.^ 



Ethnic Groups 

As shown in the first set of rows in Exhibit 4.15, among white, black, and Hispanic women 
whites were the only ones to experience a statistically significant earnings impact — with an 
estimate of $723 over the 1 8-month follow-up period. Neither the $45 7 earnings gain of black 
women nor the $-414 loss of Hispanic women was significantly different fi-om zero. 

The estimates for the three ethnic groups are not significantly different from one another 
at the conventional 90 ptsrcent confidence level. They are, however, significantly different at 
the 80 percent level. Thus, it seems likely that the difference in estimated impacts among these 
three groups reflects a real difference in program effects, not just sampling error. 

Differences in program impacts among women in different ethnic groups could reflect 
differences in the JTPA services they received. But these differences in impact could also be 
due to any of a large number of differences among whites, blacks, and Hispanics — for 
example, differences in personal characteristics, such as educational background, work 
experience, and family situation or differences in the sites in which they were living, such as 
unemplo>'ment rates, industrial composition, or availability of public transportation. 



28. Ail of the subgroup impact estimates in Exhibit 4.15 are based on ordinary least squares regressions 
on a pooled sample of all adult women, with the ticatmeni indicator interacted with the defining characicrisiic 
of the subgroup and (as appropriate) site or site and service strategy. This approach allowed direct calculation 
of the F-lesl for differences in impacts among subgroups in each panel of the exhibit. Subgroup impacts were 
also estimated on samples containing only the subgroup of interest; in general, these estimates differed little 
from those based on the pooled regressions. 
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Exhibit 4JS Impacts on Total IS-Month Earnings: JTPA Assignees, by Selected Key Subgroup 



Key subgroup, 
defined by: 



Ethnicity 

While, non-Hispanic 
Black, non-Hispanic 
Hispanic 

F-lcil, diffcicocc among subgroups 

Barriers to employment Jin iulic) 
Receiving cash welfare 
No cash welfare 

F-tect, difference belween subgroups 
No high school diploma or 
GEO certificate 
High school diploma or 
GED certificate 

F-le«l, difTcrcncc between subgroups 
Worked less than 13 weeks 
in past 12 months 
Worked 13 weeks or more 
in past 12 months 
F-lc«l, difference between subgroups 

Number of barriers 
None of the above 
One of the above 
Two of the above 
All three of the above 
F-lcsi, difference among subgroups 

Work histories 
Never employed 
Earned < $4 hourly in last job 
Earned $4 or > hourly in last job 

F-lcsl, difference among subgroups 
Employed up>on application 
Not employed upon application 

F-icsi, difference belween subgroups 
AFDC history 
Never AFDC case head 
AFDC case head less than 2 years 
AFDC case head 2 ycais or more 

F-test, difference among wbgroups 



Sample 
size' 

(2) 



Control 
mean 

a) 



3,541 
1,981 
744 



2,446 
3,500 



1,731 
4,316 

3,022 
2.622 



1,361 
1,655 
1,435 
4S8 



927 
1,924 
2,864 

955 
5,470 



3,104 
1,1 U 
1,773 



$ 8,007 
6,829 
6,775 



5,492 
8.965 



Impact in $, adjusted for 
sample distribution across- 

Sites and ser\icf 
Impact, in S Sites strategies 
(3) (4) (5) 



6.072 
8,064 

5,555 
9.956 



10,971 
7,950 
5,756 
3,703 



4,035 
6.901 
8,720 

10,497 
6.939 



8.515 
7.732 
5,555 



723*** 


$ 624** 


S 627** 


457 


AIT 






81 


55 


n.s. 


n.s. 


n.s. 


387 


406 


419 


697*** 


664** 


647** 


n.s. 


n.s. 


n.s. 


416 


445 


438 


681 


665*** 


679*** 


n.s. 


n.s. 


n.s. 


511** 


551** 


5 /I" 


668** 


600* 


561* 


n.s. 


n.s. 


n.s. 


909** 


814* 


737 


802** 


775** 


819** 


379 


392 


372 


-213 


-86 


-129 


n.s. 


n.i. 


n.c. 


568 


623 


633 


535* 


520* 


540* 


734** 


704** 


705** 


n.s. 


n.s. 


n.s. 


777 


657 


666 


509*** 


522*^^* 


535*** 


n.s. 


n.s. 


n.s. 


673** 


675** 


693** 


437 


459 


466 


570* 


542* 


525 


n.s. 


n.s. 


n.s. 



(Cotutntud) 
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Exhibii 4. IS Impacts on Total IB-Monlh Earnings: JTPA Assignees, by* Selected Key' Subgroup 
(continued) 



Impact, in adjusted for 
sample distribution across: 



Key subgroup, 
aejinea oy. 


Sample 
size 

(1) 


Control 
mean 


iwwlf?OCl p «/s ^ 

(3) 


Sites and service 
Sites strategies 
(4) (5} 


JTPA required for welfare, food 












stamps, or WIN procram ' 






% 529 


% 412 


S 385 


Ye« 


lea 


$ 5.016 


No 






^ /O 


495** 


501 


F-tcat, differeoce bctwcco tubg roups 






&.S. 


n.s. 


n.s. 


Household composition 








485 


453 


No spouse or own child prcient 


1.270 


8,035 


533 


Own child under age 4, 










550 


no apouie, present 


1.106 


A lis 




553 


Own child, none under 4, 












no apouie, present 


1,801 


7,ooi 


o/y 


685* 


Spouse present, with or 










832 


without own child 


1.323 


7.527 


818** 


860** 


F-te«t, diffetence smoog subgfxwps 






n.s. 


U.S. 


n.s. 


Family income in past 12 months 










462** 


$6,000 or less 


3.861 


6,646 


480** 


488** 


More than S6.000 


2,107 


9.379 


586* 


529 


572 


F-teat, diffcfTDce betweeo subfrcMipi 






O.S. 


n.s. 


n.s. 


Living in public housing 










-436 


Yes 


755 


6.370 


-466 


-420 


No 


5,582 


7.656 


675*** 


670**-' 


674*** 


F-teit, diffeieace between tubfrcMips 






* 


* 


* 


Age at random assignment 










580* 


22-29 


2,830 


7.265 


611* 


596* 


30-44 


2.825 


7.805 


440 


451 


436 


> 44 


819 


6.897 


643 


647 


780 


F-leat, dtffcfence among tubgroupt 






n.s. 


n.s. 


n.s. 


Recommended for JSA only 












Yes 


312 


6,815 


824 


1,136 


n/s 


No 


6,162 


7,490 


512*** 


497**. 


n/a 


F-tc«l, difference bclwcea subgroups 






n.s. 


n.s. 


n.s. 


Full sample 


6,474 


7,488 


539*** 


539*** 


539*** 



Sources: EstinM hmtd on Fint FoUowHip Surv^ tv^iooMB and CMBiflg* dsu fm aiaie UI agmcin. 

Notes: Btiimta are regmeion-adjustad to conuol for difTotnoea in baseline chtnctoislios bctweoi Uk traatmcnt grot^ and oontral 
gf<M^\ see Appmitx D. CoMtrol mans were not Rgmion adjusttxl. Sample sizes for muoially cxcluiive siiigroupg within s pwid do 
not Dcccassfity stan to the SM^k size for tbe target gnmp as a wfaoic, becMise persons io omiaed subgroi^ or with misting data on the 
vtriable used to define the subgrot^ are excluded. 
TfiauneMuidconifDlgrouptoonibned. 

b. AFDC. Goienl AasisUMce. or ether welfare except food stamps. 

c. WIN is llw fsdoml Work Inccnuve prog rsm. 

* Stsltttkmlty Mfwrxsnl sc the .10 kvel. st the .05 kvel. *** at the .01 kvcl <F*lest or two-itiled to«eM): 'b.s.* mtmm the F-lMt for the 
diflcfmoe in kiyacts between or smong the subgfoupi in the ositgory is not statistkaliy signiBanL 
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As shov.-n in Exhibit 4.15, the adjustments for differences among ethnic groups in their 
distribution across sites and across sites and service strategies had little effect on the estimated 
impacts, with one exception. The adjustments for the distribution of Hispanic women across 
sites and across sites and service strategies changed the estimated effect from substaniialK' but 
insignificantly negative to insignificantly positive, eliminating the initial weak evidence that 
impacts varied with ethnicity. Hispanic women were much more concentrated in a small 
number of sites than adult women overall. Once these locational differences are removed, any 
remaining differences in JTPA service mix or in the background and personal characiensiics 
associated with ethnicity were not of themselves sufficient to create detectable variations in 
impact. Thus, the estimated negative effect on Hispanic women was associated with the 
geographic distribution of this subgroup. However, given the extreme concentration of 
Hispanic women in a small number of sites, we cannot reliably distinguish negative effects on 
Hispanic women as an ethnic group from more general negative site effects on all women in 
one or more of the sites in which the Hispanic women were concentrated. 



Subgroups Facing Selected Barriers to Employment 

Women who apply to JTPA may face a number of barriers to employment. Those receiving 
welfare face the financial disincentive posed by the loss of welfare and Medicaid benefits 
should they become employed. Some of these women may also face the barrier of finding 
adequate child care. Other women applying to JTPA may have limited education or limited 
recent work experience. To measure the success of JTPA in preparing participants to 
overcome these barriers, we estimated impacts on women who were receiving cash welfare 
benefits (AFDC, General Assistance, or other welfare except food stamps), upon their 
application to the program; women without a high school diploma or GED certificate upon 
application; and women with less than 13 weeks of employment in the 12 months preceding 
their application.^' 

The mean earnings of the control groups in column 2 of Exhibit 4.15 indicate that these 
barriers were indeed serious obstacles to employment for the women in our sample. Control 
group members in each of the three subgroups facing these barriers to emploN-ment earned 
much less over the 18-month follow-up period ($5,492, $6,072, and $5,555, respectively) 
than those facing none of these barriers ($10,971). 

For comparison purposes the exhibit shows the estimated impacts both on women facing 
each barrier to employment and on those who were not facing that barrier. The program had 
no significant effect on the earnings of women who were receiving welfare when the\' applied 

29 Although the precise measures may differ, the three dimensions of welfare recipiencv, limiiod 
education, and limited work experience arc important and common focal pomts in much of the literature and 
agency reporting on JTPA. 
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to JTPA or on women with less than a hij h school education, but it did significantly increase 
the earnings of their counterparts: by $69 / for women not receiving welfare upon application, 
and by $681 for women who had a high school credential upon application. 

Nevertheless, there was no significant difference between the impacts on these subgroups 
and the impacts on their counterparts who did face the barrier in question (as indicated by the 
F-test result.) That is, there was no significant difference between the impacts on women 
receiving welfare upon application and women not receiving it, or between the impacts on 
women with a high school credential and those without one. Similarly, although we are 
confident there was a positive effect on the earnings of women with limited recent work 
experience, we cannot say this effect was greater than that on women with more employment 
experience in the year preceding their application. 

Defining subgroups in terms of the presence or absence of specific barriers to emplovTnent 
does not fully distinguish subgroups in terms of the overall difficulty they faced in becoming 
employed. For example, the subgroup with limited education includes some women who were 
also receiving welfare, some with limited recent work exp)erience, and some with both of these 
other two barriers. We therefore also estimated impacts on women with none, one, Uvo, or all 
three of these barriers. As shown in the next four rows of the exhibit, this categorization 
provides clearer distinctions among the subgroups, in terms both of the difficulty of becoming 
employed and the program impact. 

The control group earnings levels indicate that in the absence of the program women facing 
only one of the three barriers earned more than twice as much ($7,950) as women facing all 
three barriers ($3,703) and about a third more than those facing two of the three ($5,756) . And 
all three subgroups feeing at least one barrier to employment earned much less than the 
subgroup facing none of the barriers ($10,971). 

When categorized by the number of barriers to employment they feced, the only women 
to experience a statistically significant earnings impact were those facing zero or one barrier. 
The estimated impact in these subgroups was $909 and $802, respectively, or about 8 percent 
and 1 0 percent of the corresponding control group means. About half of the women feeing a 
single barrier had limited recent work experience; the remainder were about evenly divided 
between those receiving welfare upon application to JTPA and those with limited education 
(not shown in the exhibit). Thus, these results are cons istent with the estimates in the preceding 
sets of rows in the exhibit showing a significant impact on the earnings of women with limited 
work experience, but not significant impacts on women in the other two subgroups. 

Again, however, the impact estimates for the set of subgroups defined by number of 
barriers to employment were not significantly different from one another. We can therefore 
be confident there were positive impacts on those facing one barrier and on those facing no 
barrier, bu t wc do not know if those impacts were larger than the impacts on the other two subgroups . 
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Subgroups with Different Work Histories 

The next panel in th e exhibit shows that Jl PA significantly increased the earnings of women 
who had worked before by $535 to $734, including both women who worked for low wages 
(less than $4 hourly) on their most recent job and higher wage workers. There was no 
significant effect on the earnings of women who had never worked before, although the 
estimated impact on their earnings differed little from those on the subgroups who had worked 
before. Not surprisingly, there was a significant impact on the earnings of women who were 
not employed upon application to JTPA, who formed the overwhelming majority of the sample 
(see column 1). The impact on women who were employed was not statistically significant. 
Within both sets of subgroups, however, the estimated impacts across subgroups were not 
significantly different fi'om one another. 



Subgroups with Different Public Assistance Histories 

JTPA had a statistically significant, positive effect on the earnings of women who had never 
been an AFDC case head. For this subgroup the estimated impact on total 1 8-month earnings 
was $673, or about 8 percent of mean earnings in the corresponding control group. Although 
there have been several experimental studies of employment and training programs for AFDC 
recipients,'^ to our knowledge this is the first experimentally based estimate of training 
program impacts on women who are not welfare recipients. 

Among women who had been AFDC case heads before applying to JTPA, there was no 
significant impact on the earnings of those who had been case heads for less than two years, 
but women who had been case heads for two years or more experienced statistically significant 
earnings gains of $570 over the 18-month period. There was, however, no statistically 
significant difference among the estimated impacts for the subgroups defined by AFDC 
history. 

It should be noted that these last two subgroups include all women who had ever been an 
AFDC case head prior to application. They differ, therefore, fi"om the subgroup receiving 
AFDC at the time of application, which was discussed above in connection with barriers to 
employment, although there is a substantial overlap between the two groups.^' 

30. Se£, for example, Enns, Bell, and Flanagan (1987) and Gueron and Pauly (1991). 

3 1 . Among women vAio had been AFDC case heads for two years or more before applying to JTPA, 79 
percent were receiving AFDC at the time of their application. These cases accounted for 57 percent of women 
in families receiving cash welfare at the time of application. Among women who had been AFDC case heads 
for less than two years before applying to the program. 61 percent were AFDC case heads when they applied 
These cases accounted for 34 percent of women in families receiving cash welfare upon application (Tiie 
remaining 9 percent of those in families receiving cash welfare upon application cither were receiving cash 
welfare other than AFDC or were not the AFDC case head.) 
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There was no significant program effect on the earnings of the smail group of women \n ho 
reported that they were required to apply to JTPA as a condition of receiving welfare or food 
stamps or as part of the Work Incentive (WIN) program. 

i 

Subgroups from Different Types of Households 

JTPA had relatively large, significant impacts on the earnings of both women who lived with 
older children (age 4 and over) but no spouse ($679) and women who lived with their husband 
with or without a child present (SS 18). The program did not have f significant effect on the 
earnings of women who lived without a spouse or child or of those with a young child but no 
spouse present. The estimated effects for these four subgroups were not, however, 
significantly different fi-om one another. 

The estimated effect on the earnings of women whose family income was $6,000 or less 
in the year before their application was a statistically significant $480, or about 7 percent of 
the corresponding control group mean. The effect on women from higher income families 
($586) was also significant. The difference in impacts between these two groups was not 
statistically significant. 

The program had no significant impact on the earnings of the relatively small subgroup 
of women living in public housing upon their application, llie estimated impact for this 
subgroup ($-466) was significantly less than that for all other women ($675), suggesting that 
women in public housing benefited less from the program. 



Age Groups 

JTPA had a significant impact on the earnings of only the youngest adult women in the sample, 
those ages 22 to 29 at ranoom assignment. The estimated effect for this subgroup ($611) did 
not differ significantly from the impacts for older women, however. 

Women Recommended for Job Search Assistance Only 

Finally, to measure the effect on this target group of a low-intensity approach that has 
frequently been used with welfare recipients, we estimated the impact on the earnings of those 
women for wl.om program intake staff recommendei job search assistance (JSA) only.^- 

32. Estimated impacts for the three major service strategy subgroups were presented earlier and arc 
therefore not repeated here. As with the other subgroups in Exhibit 4. 1 5» we did test for significant diOerences 
in impact among the three service strategy subgroups, and estimated impacts adjusted for di (Terences in 
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These women comprised about a quarter of the other services subgroup." This subgroiip was 
too small to estimate the impact very precisely. Thus, the relatively large estimated impact 
for this subgroup is not statistically significant, indicatingthat itmay bcdue to sampling error. 



A Summary of Impacts, in the Context of Previous Research 

This last section summarizes the impacts of JTPA Title II-A on the earnings, employment, and 
high school attainment of adult women, with reference to results from previous experimental 
research where they are comparable to our own. Exhibit 4. 16 displays the principal findings 
in this chapter. We consider our findings for the target group as a whole, summarized first, 
as benchmarks for drawing conclusions firom our more detailed findings for service strateg\- 
subgroups and other key subgroups, which we will then examine in turn. 



Adult Women Overall 

The estimates presented in the first three sections of this chapter document a consistentK' 
positive impact on the earnings and employment of adult women. The $539, or 7.2 percent, 
earnings gain attributable to the program over the follow-up period as a whole was primariK- 
due to program impacts on the percentage of the treatment group who were employed at some 
time during the period and on earnings per hour worked. There was little or no effect on 
average weeks worked by those who were employed or on hours worked per week employed. 
Thus, JTPA appears to have increased the probability that women would be employed, and 
raised the houriy earnings of those who worked, but it did not lead to steadier employment or 
more fiill-time work. 

Since not all women in the treatment group actually became enrolled in JTPA, impacts 
on the average earnings of all JTPA assignees understate the impacts on the earnings of those 
who were enrolled. Although we cannot estimate impacts on cnrollces* earnings with 
certainty, if we can assume the program had no effect on the earnings on nonenroUecs, then 
the inferred impact on earnings per enroUee was $873 over the entire follow-up period. 



distribution across sites. The estimated impacts for the three service strategy subgroups were not significantly 
diflcrcnt from one another, and adjustment for dilTcrences in distribution among sites had little eflcci on the 
imp-^ct estimates. 

33. See column 1. As explained in Chapter 2. it was only in the other service subgroup thai sample 
members could br recommended for job search assistance alone. 



184 



JTPA 18-MONTH IMPACTS / ADULT WOMEN • 123 



Exhibil 4,16 Summary ofEstimaxedJTPA Impacts on Earnings and Employment 
over the Full Follow-up Period: AduU Female JTPA Assignees in 
the 18-Month Study Sample, by Service Strategy Subgroup 



Impact per 
assignee on: 


All 
subgroups 
(I) 


Classroom 
training 
subgroup 
(2) 


OJT/ 

subgroup 

(3) 


Other 

subgroup 
W 


Earnings 








% 457 


InS 


$ 539*** 


$ 398 


$ 742** 


As % of control mean 


7.2% 


6.2% 


8.6% 


5.7% 


Percentage employed* 


2.1%** 


1.3% 


2.4% 


4.3%* 


Weeks worked 










In weeks 


1. 1 


-0.1 


0.7 


3.7** 


As % of control mean 


2.8% 


-0.3% 


1.6% 


9.9% 


Hours worked 










In hours 


52 


-30 


97* 


79 


As % of cobitrol mean 


3.7% 


-2.5% 


5,9% 


5.7% 



Source*: E*iirtAc« bwed on Bm Fdlow-«p Survey te^onies and eitnings dti« from «jU UI agencies. 
Nous: SwT^Ic lixc, cUssroom training «*bgtoup: asagnecs « 1,916, comtol group » 931; OJT/JSA 
subgrtx4>: as&gnees » 1,538, cortirol group » 749; other service* »bgtoup: ustgnoes » 922, cocaiol group 
« 418. EsUnwtc* art r^fesMOfhM$u«fid to oonirol (ix difference* *^ 
ireatmcni group and comrol group; aee Appendix D. 

«. Ai any tirm during the 18-moc«h foMown^) period. Tlic impact i* mea«if«d in percentage poinu. 

♦ SiaUftically wgnificart « the .10 level, « the .05 level, at the .01 Icvd (two-tailed ttti). For the 
inpacu on earnings, week* woriced, and houn worked, the iignificance level of each c«im«e 
expressed 'as % of control mean" is the same as thM of the oorrtapoodir^ estimate ejqpreaaed in doHars, 
in weeks, or in hours. 



Women Recommended for Each Service Strategy 

The estimated impacts on earnings and employment reflect the increment in employment and 
training services received by the treatment group as a result of its having access to JTPA 
Patterns of service receipt were markedly different across the three service strategy sub- 
groups, but in each case a very high percentage of the women who enrolled in JTPA received 
one or both of key services provided in the strateg\' recommended for them. In the clasr,room 
training subgroup 88.8 percent of JTPA enrollecs received classroom training in occupational 
skills, basic education, or both. In the OJT/JSA subgroup 87.8 percent of cnrollees received 
on-the-job training, job search assistance, or both. And in the other services subgroup 82.3 
percent received job search assistance, miscellaneous services, or both. 

Thus, the service recommendations of program intake staff did create three distinct groups 
in terms of the services their members actually received . The three service strategy subgroups , 
then, constitute a reasonable test of three distinctly different service strategics. 
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But the impact of JTPA d-pends not only on the JTPA services provided to women in the 
treatment group. It also depends on the services they would have received from other 
providers had they not had access to JTPA, which we measure by service receipt in the control 
group. 

Direct evidence of the control group's receipt of non-JTPA services was a vailable only 
for classroom training in occupational skills and basic education. In the classroom training 
subgroup treatment group members were more likdy than control group members to receive 
classroom training in occupational skills (48 6 percent versus 28.7 percent) and basic 
education (1 1.2 percent versus 7.5 percent). 

We can also be confident that JTPA substantially increased the receipt of on-the-job 
training in all three subgroups, since this service is seldom provided by non-JTPA programs. 
We were unable to measure the extent to which JTPA increased the receipt of job search 
assistance or miscellaneous services. 

While access to JTPA substantially increased the incidence of receipt of occupational 
skills training, basic education, and OJT, the treatment-control differential in hours of 
occupational skills trziidngper assignee for women in the classroom training subgroup was 
only llOhours. Similarly, women in the OJT/JSA treatment group received only 104 hours 
more OJT per assignee than their control group counterparts. These relatively small 
differentials reflect the fact that not all treatment group members received these services and, 
in the case of occupational skills training, that a nonnegligible proportion of control group 
members received services. 

These service differentials led to significant increases in educational attainment 
for women in the classroom training and other services subgroups, who were those most 
likely to receive occupational skills training or basic education. In the classroom 
training subgroup 29 percent of the high school dropouts in the treatment group 
achieved a training-related high school diploma or GED certificate, whereas only 
11 percent of the corresponding control group did. In the other services 
subgroup 17 percent of the high school dropouts in the treatment group attained a 
training-related high school credential, whereas 10 percent of the corresponding 
control group did. Across all three service strategy subgroups, the program nearly 
doubled the high sdiool attainment rate among dropouts, from 1 1 percent to 19 percent. 

The estimated effects of JTPA on the earnings and employment of women in the three 
service strategy subgroups were very consistent with the patterns of effects one would expect, 
both across the subgroups and over time within each one. For example, women in the 
classroom training subgroup experienced a significant earnings loss in the first quarter after 
random assignment, ref.ecting their investment of time in training rather than working. But 
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throughout the rest of the follow-up period they received progressively nsing returns on this 
investment, wth impacts on earnings per assignee becoming statistically significant in the 
fifth quarter and reaching a significant $ 1 88 in the sixth quarter. These offsetting losses and 
gains summed to a statistically insignificant 18-month earnings gain of $398, or 6.2 percent 

In contrast, women in the OJT/JSA subgroup experienced positive earnings impacts 
beginning in the first quarter and totaling $742 (8.6 percent) over the follow-up period. The 
estimated impacts were fairly constant inthe $109 to $144 range and were significant in five 
of the six quarters. This pattern is consistent with an employment strategy that attempts to 
put women to work immediately, cither in an on-the-job training position or in a regular job. 

The pattern observed for the other services subgroup, on the other hand, was one of 
relatively quick, but short-lived, effects on earnings. The estimated earnings gain of $220 in 
the third quarter after random assignment sharply declined and became statistically insignifi- 
cant in the later quarters. This pattern is consistent with the mix of relatively brief and 
nonintensive services provided to these women. The total 18-month earnings gain for this 
subgroup, $457 or 5.7 percent, was not statistically significant. 

Although the effects on total 1 8-month earnings for the classroom training subgroup and 
the other services subgroup were about tlie same size, the trends in the effects over time suggest 
that this overall comparison may be misleading. Because the effects of the other services 
strategy appear to have peaked and begun to diminish some time before the end of the follow- 
up period, whereas the effects of classroom training— and OJT/JSA— were growing or stable, 
it seems likely that the subgroups recommended for these latter two strategies will witness 
larger long-term effects than the other services subgroup. This issue will be addressed in our 
forthcoming final repoi,. 

The patterns of effects on the components of earnings are also consistent with the 
divergent nature of the three service strategics. In the classroom training subgroup the 
earnings gain over the entire follow-up period was largely the result of a positive impact on 
earnings per hour employed, which offset small decreases in the duration of emploNinent spells 
and the extent of fiiU-time work. For this subgroup the program had only a small positive 
effect on the percentage employed. This pattern is consistent with the view* that classroom 
training involves an initial investment of for£ ^ne employment and earnings that then leads to 
higher rates of pay. 

In contrast, the overall earnings gain in the OJT/JSA subgroup was more evenh* 
attributable to increases in the employment rate, hours worked per week worked, and hourly 
earnings when employed. This pattern is consistent with a program strategy of placing 
participants immediately in OJT slots or regular jobs that offer more fiiU-time work and paN* 
higher wage rates than those the participants would have found in the absence of the program 
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The impact on earnings in the other services subgroup was apparently due primariK' to 
an increase in this group's employment rate, which peaked in the second and third quarters. 
This pattern is consistent with a strateg>' of relatively short-term, low-intensity services aimed 
at enabling participants to find jobs as quickly as possible. JTPA seems to have been 
successful in carrying out this strategy for members of this subgroup. But it does not appear 
that the jobs they'found were much better, in terms of their hours per week or rates of pa> , 
than the jobs they would otherwise have gotten. 

Despite the apparent coherence and consistency of the story these results tell, it must be 
remembered that the subgroups differed not only in the nature of the services recommended 
and received but also in the kinds of women served. For example, intake staff appear to have 
recommended more job-ready women to the OJT/JS A subgroup. This means that differences 
in impacts rcross the subgroups cannot be attributed solely to differences in services. Thus, 
while we can identify those service strategies that were working, or not working, for the groups 
of women they actually served, we cannot say whether the impacts for any subgroup of women 
could be improved by substituting a different mix of services for the one the subgroup 
members actually received. 

Furthermore, the costs of these three service strategies were likely to be quite different. 
Differences in impacts therefore may not accurately reflect differences in cost-effectiveness. 
Our final report will present a benefit-cost analysis that takes these cost differences into 
account. 

To provide a broader context vwthin which to assess the impacts reported here. Exhibit 
4.17 shows the impacts estimated in previous experimental evaluations of emplovonent and 
training programs serving women.^^ In all cases these programs served only AFDC recipients 
T applicants; we know of no experimental studies of programs serving other women. Thus, 
the participant populations in these studies are not fully comparable with the sample \n the 
National JTPA Study. Moreover, unlike in JTPA, participation in many of these previous 
programs was mandatory, with potential sanctions for nonparticipation. 

The services provided varied substantially across these programs. In the discussion here 
we will therefore compare the impacts of each program to the impacts of the JTPA service 
strategy that involved the most comparable services. Note that the impact estimates shown 
have been calculated as average quarterly impacts during each of the first three years 
following random assignment (columns 3 through 5). The year 1 impact estimates showi) for 
JTPA are averages of the estimates for the first four quarters after random assignment; the 
year 2 estimates are averages of the fifth and sixth quarter estimates only. 



34. The studies included are only those with final impact results available. Theestimates from the previous 
studies have been converted to July 1989 dollar values. 
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ExhihU4.J7 Btunat^ Impacts on the Average QuantrfyEarnintscfAdktitWomm: The JTFA J S-Motuh Impact Analysis and 
Previous Expcrimmtal SotMes 



Quarterly $ impact per ireatmeni group 
member (% tmpaci in parentheses. 



(Year evaluation 
befon) 


Treatmmt grottp 

(1) 


Program services 

(2) 


Year J 

(3) 


year 2 * >rj' i 
r-^i ^* ■ 


Ntiiooal JTPA Study 
(1987) 


Low-ifioocae womeo, 
34% feodvHic AFDC, 
74% wHh diiU of 
pwticipfttioo 
vduDUuy 


CiMtfoom Miaiof 
■limtciy: 

ooctipttiooiJ akiik 
iiminiBf , hemic 
«ducacioa^ 


S16 

C2%) 


S 166»» 

(13%) 






Orr/JSA lUmlccy: oo- 
Ibe'job timiniii|. job 
Mftrcfc •Miataooe 


s n6»»» 

<«%) 


S 140»» 

(9%) 






Other tcrvion fttntcfy: 
job tarcb aHiauooe. 


S103 
(S%) 


$22 
(1%) 






All tknc MiMccics 


$7l»» 

(6%) 


$129»»» 

(9%) 



Work-w«lf«rc 
(WIN) AudiM: 
Low-inieaiiQr 

(1978-19 J5)^ 



Moderate intcn* it>* 
(19t2.1985) 



on* prof fwni ' 
(1983-1984) 



AFDC appUoittU 
and recipkaU;' 
ptfticipfttioii 



Job tarcb Miitlincc, 
unpaid wofk cxpcri* 
ciKC occxiptiooal 

heakCi 



AFDCappIicanu 
and racipicati wUh 
ynungH duld af e 6 
or older panicipaftion 
mandalofy 

AFDC recipicms with 
child of any age; 
partipatioo vokiotafy 



Job Karcfa aif isuncc. 
unpaid work expcri- 

lakillt 



tniaiaf, baitc aducttioo 

Job-read iDCM irauiini. 
uapaid woik cipcrienoe. 
oo-thfr>job tiaiDiof 



S 3 to 138* 
(l%»23%) 



$4ru !eo*» 

(10V-.21%) 



$31^ 
(8%) 



$65iol32»» 
(23%. 20%) 



$1I8'»» to 180» 
(17%. 29%) 



$165»io249»» 
(14%. 38%) 



S 76» 10 126^^ 

(11%. 18%) 



(17%) 



265-' 
(34%) 



Other work-wilfare aludia 
AFDC Homaoakcr- 
Home Healih Aide 
DoTMiMtimtioiii* 
(1983) 

Nuioaal Suppoitad 
Woik DcflMMitfttiioB"* 

(197«) 



AFDC laciptcpu with 
chiU of any af e; 
partjpation voluMtar>*' 



AFDC laripitu for 30 
of poat 36 aioaiki. with 
chiU of any af e; 
paiticipatioe vohinury 



OccupaiionaJ akilk 
ininiaf and praAicwn 
iti '. iie care, up lo 12 
moAtha aubtidized 
empbymeat 

Stfuouiiad. paid woik 
eiycrkoo e for up to 
12 I 



S 113 lo 1.058* 
(15%. 271%) 



S 1.847» 

(-)* 



$147** to 669»»* S 39 ;o 573»»» 

(16%. 76%) (137.78^) 



S 395»»» 

(36%) 



(23%) 



Sources: Reviewed in Gucrofi and Pauly (1991) 

Notes: Dm rtportad caiiimiM ^e bae» convened to July 1989 doUan. The * % impact* i« the *S impact* ei^reaMd as a 
perocataf e of the cootrol fnnip mean. 

(eontinuai) 
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The first panel of the exhibit suitunarizes the impact estimates presented earlier in this 
chapter for adult women in each of the three JTPA service stratcg>' subgroups and for the 
target group as a whole. The next panel presents results from eleven studies of state Work 
Incentive (WIN) programs and WIN dcmonstiation work-welfare programs conducted in the 
late 1970s and early 1980s. The first two sets of work-welfare programs, those offering low- 
and moderate-intensity services, were mandatory for certain groups of AFDC applicants and 
recipients. In the first set seven low-intensity programs offered primarily group and mdividual 
job search assistance and unpaid work experience. The two moderate-intensity programs in 
the second set offered these services as well as occupational skills training and basic education 
for up to a quarter of the participants. And the two demonstrations in the third set emphasized 
on-the-job training for volunUry participants already receiving AFDC. 



a. The estimates ftom the National XT? A study (fint panel) in thii column are for the firtf two quarters only. 

b. "Basic education" includes Adult Basic Education (ABE), high school or General Educational Development (GED) 
pirpaxation, and English as a Second Language (ESL). 

c. "Miscellaneous" includes assessment, job-icadineas training, customized tiaining, vocational exploration, job shadowing, and 
tryout employment, among other services. 

d. Included the following progranis (net cost per treatment group member, in nominal dollars except where so noted, in 
parentheses): Arkansas WORK Pro^nm ($118), Cook County (lUinois) WIN Dcmonsirttion ($157), LouisviUe WIN 
Uboratory Experimenu ($136. individual job search assistance, $230, group job search aasi^ance, converted to 1985 dollars). 
San Diego En^loyment Prq)aration Progrmm/Expcrimcntal Work Experience Program ("San Diego 1") ($636), Virginia 
Employment Services Program ($430), and Wc*^. Virginia Community Work Experience Program ($260). 

e. Targeted to AFDC applicarts and itcipienU with youngert child age 6 or older, except in Arkansas (child 3 or under); 
LouisviUe (child any age); San Diego I (AFDC applicanU only, excluding those who were refugees, en^Ioyed, or 
monolingual in a language other than English or Spanish); Virginia (excluded previous program participants and current 
full-time participants in other education or training wrvices); and West Virginia (excluded those cnyloyed or in full-time 
education or training). 

/ Impact estimates for years 2 and 3 available only for Arkansas, Louisville (individual job search assistance only), and Virginia. 

g. Included the following programs (net cost in nominal dollars per trcainr^ent group member in parentheses): San Diego 
Saturation Woik Initiative Model (SWIM) ($919) and Baltimore Options Program ($953). 

h. Impact estimates for year 3 available only for Baltimore. 

I. Included the foUowing programs (net cost in nonunal dollars per treatment group member, including wage paymenu for 
subsidized employment, in parwtheaes): Maine On^hcJob Training Program ($2,019) and New Jerecy On-the-Job Training 
Program ($787). 

j. Impact estimates for years 1 and 3 available only for Maine. 

k. Included demonstrations in the following states (not cort in nominal dollars per treatment group member, including wage 

payments for subsidized employmei*, in paiw^hcaea): Arkansas ($5,367), Kentucky ($6,955), New Jersey ($7,067), New York 
(S3,651), Ohio ($5,977), South Carolina ($7,907). and Texas ($6,267). 

/. In South Carolina targeted to AFDC recipienu with youngest child age 1 1 or older. 

m. Included demonstrations in aeven sites, with a net cost per tr^tmcnt group member of $8,093 (1976 doll »rs), or $ 10,23 1 

and thereby receiving heavily subsidized earnings during that year. 
* Statistically significant at the . 10 level, *• at the .05 level. *•* at the .01 level (two-tailed te*). 
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The low-intensity programs are probably most comparable to JTPA's other sen'ices 
strategy, while the OJT demonstrations arc most comparable to the OJT/JSA strateg\'. The 
moderate-intensity programs probably fell between the JTPA classroom training subgroup 
and the JTPA other services subgroup in terms of the intensity of services. 

The impact of the JTPA other sendees strategy appears quite comparable to that of the 
low-intensity work-welfare programs in year 1 but not in year 2 . In both dollar and percentage 
terms the work-welfare programs had much larger impacts than JTPA in the second follow- 
up year, effects that continued throughout the thrcc-ycar follow-up periods for the studies. In 
contrast, as shown earlier in this chsqjtcr (Exhibit 4. 1 2), earnings gains from the JTPA other 
services strategy had ended by the fourth quarter after random assignment. 

The impacts of the OJT work-wclfere programs also appear to have been larger after year 
1 than that of JTPA's OJT/JSA strategy. In dollar terms this component of JTPA had a much 
larger impact than the two WIN demonstrations in year 1 and a smaller impact in year 2. In 
percentage terms, on the other hand, JTPA and the work-welfere OJT programs for which 
impact estimates are available had the same impact in year 1, but work-welfare had much 
larger impacts than JTPA in year 2. Overall, the order of magnitude of the impacts appears 
to have been similar for the three programs. 

None of the pre lous employment and training programs for women that have been 
rigorously evaluated with experimental designs offered a service mix that is closely compa- 
rable to the JTPA classroom training strategy. As noted above, the two moderate-intensity 
work-welfere demonstrations most closely resembled the JTPA classroom training service 
strateg\\ but the services they offered were somewhat less intensive. The estimated dollar 
impact of the JTPA classroom training strategy was quite comparable to those of the 
moderate-intensitj^ work-welfare programs in the second year, but the JTPA impact was much 
smaller in percentage terms, reflecting the lower earnings levels of control group members in 
the work-welfare programs. 

The AFDC Home.r'-xer-Home Health Aide Demonstrations, shown in the bottom panel 
of Exhibit 4. 17, offered four to eight weeks of classroom training but also provided up to a 
year of subsidized employment, including substantial on-the-job training. SimilarU', the 
National Supported Work Demonstration offered up to 12 months of subsidized emploNonent. 
Thus, it should not be surprising that both of these programs had substantially larger estimated 
impacts than any of the JTPA service strategies. It should also be noted that the estimated 
impacts shown for these demonstrations include substantial amounts of wage subsidies paid 
by the demonstrations in the first two years. 

In general, then, to the extent that comparable impact estimates are available from 
rigorously evaluated programs for women, they tend to be similar to or slightly more favorable 
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than those found here for JTPA. These comparisons must be viewed v\ith caution, ho\ve\ er, 
because both the services and the participants in these earlier programs differed in important 
ways from those in JTPA. In particular, all of the other programs served only AFDC 
applicants or recipients, whereas only a third of the adult women in our 18-month siud\ 
sample were receiving AFDC upon application. Moreover, participation was mandator\ for 
AFDC recipi ents in some of these programs . There were also differences in the extent to whi ch 
treatment group members in the various studies actually received any program services, with 
service receipt rates ranging from a low of 38 percent in one of the work-welfare studies to 
97 percent in Supported Work. (Recall that 65 percent of the adult female treatment group 
members in our 18-month study sample were enrolled in JTPA services.) Finally, a full 
comparison with the JTPA sample must await the longer term follow-up and benefit-cosi 
analysis we will present in our final report. 



Women in Selected Key Subgroups 

In addition to presenting estimated impacts for the three service strategy subgroups, this 
chapter has also provided estimated impacts for a variety of other subgroups of women of 
interest to policymakers and program planners. Those results showed significant positive 
impacts on total 18-month earnings for a number of subgroups of women. Ir most cases, 
however, the differences in estimated impacts among subgroups were small, a; d in only two 
cases did these differences even approach conventional levels of statistical significance. 
Those two ca^cs were Hispanic women and the very small subgroup of women who lived in 
public housing. Neither of those subgroups experienced significant earnings gains as a resu It 
of the program, and the estimated impacts on their earnings were substantially and signifi- 
cantly less than the estimated impacts for other women (namely, white women and women not 
living in public housing, respectively). 

In all other cases, while we can be more confident that the program had a positive impact 
on those subgroups with statistically significant estimated impacts, we cannot sa\' \sith 
confidence that those impacts were larger than the ' pacts on the subgroups whose impact 
estimates were not significant. 

We also tested to see whether variations in i across subgroups were due to 

differences in the distributions of subgroups acrosa sites or sites and service strategies 
The estimated impacts were largely invariant with adjustments for these distributional 
differences among subgroups. We therefore conclude that variations in impacts across 
subgroups reflect primarily differences in the characteristics of the women 
themselves (and random sampling error), not differences in where they were living or the 
services they received. The one exception to this general statement was the estimated impact 
for Hispanic women. This difference in effects appears to be associated with the 
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geographic distribution of Hispanic women. But because of the extreme concentration of 
Hispanic women in only a few sites, we cannot reliably distinguish whether the negative impact on 
this subgroup was associated with ethnicit>' per se or with a more general negative 
eflfect on all women in one or more of the sites in which the Hispanic women were concentrated 



Adult Men: JTPA Impacts at 18 Months 



THIS chapter presents estimates of the impact of JTPA Title II-A programs at the 16 
study sites on the earnings, employment, and high school attainment of adult men over 
the 18 months following their random assignment. These findings offer the first reliable 
information about the effectiveness of Title II-A programs for men and considerably expand 
existing knowledge about employment and training programs serving economically disadvan- 
taged men in general. 

The 5,626 men in the 18-month study sample were ages 22 or older at their random 
assignment; and like the adult women in the sample, they had an average age of 33, with 88 
percent between ages 22 and 44 and less than 5 percent age 55 or older.* The men also 
resembled the women in ethnicity, with 57 percent white, 29 percent black, and 10 percent 
Hispanic. But as we might expect, the men had, on average, higher wage rates in their most 
recent job, as well as more extensive employment experience. Over 90 percent of the men 
had held a job in the past. Nevertheless, only 13 percent were employed upon application 
to JTPA. Like the women, over two-thirds ofthemenhad ahigh school credential. Incontrast, 
only 6 percent were receiving Aid to Families with Dependent Children (AFDC) when they 
applied to the program, while 27 percent were receiving food stamps — rates far below those 
of the worn en. 



1 . Among all the men in the 1 8-monlh study sample, 4,5 1 9, or 80.3 percent, responded to the First Follow- 
up Survey. The infomialion on baseline characteristics presented here is based on Background Infomiation 
For.r. rc:ponscs from all men in the 18-month study sample. 

2. See Exhibit 3.13 in Chapter 3 for more detail. 
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The Structure of this chapter repeats that of the preceding one. We first present estimates 
of JTPA impacts on adult men overall, in three sections examining, in turn, impacts on 
earnings, on emploN-ment, and on the components of earnings. Unlike the preceding chapter, 
the impact estimates throughout are based exclusively on First Follow-up Surve>- data, 
because tests of survey nonresponse bias indicated negligible nonresponse bias in the data for 
adult men. 

In the fourth section of the chapter, we turn to estimated impacts on the subgroups of 
adult men recommended for each of the three service strategies defined in Chapter 2 : classroom 
training, OJT/JSA, and other services. The fifth section gives estimated impacts for selected 
key subgroui>s of men defined in terms of individual characteristics, such as barriers to 
employment, that may have affected their ability to benefit from JTPA. Finally, in the chapter 
summary- we review the findings and place them in context with estimates from previous 
experimental studies of employment and training programs serving adult men. 

Impacts on Earnings: Adult Men Overall 

This section presents estimates of JTPA Title II-A impacts on the earnings of adult men. We 
begin by simply contrasting the monthly earnings trends of theUeatment and control groups. 
We then describe the pattern of treatment group enrolbnent in Title II-A over the 18-month 
follow-up period to draw a distinction between the in-program and post-program periods for 
the group as a whole. The section next presents numerical estimates of program impacts on 
earnings per JTPA assignee and per enrolleeand ends by discussing impacts on the distribution 
of earnings. 

Monthly Earnings Trends: Treatment Group and Control Group 

Exhibit 5 . 1 displays the trends in the average monthly eaniings of treatment group and control 
group members during the 1 8 months following their random assignment.' The exhibit shows 
that average earnings were virtually the same for the two groups during the first 3 of the 18 
follow-up months. After that point, treatment group members, on average, consistently earned 
more than their control group counterparts. 

The estimates underlying the earnings curve for the treatment group represent an outcome 
of JTPA— what the treatment group earned after its members gained access to JTPA: from 



3 Throughoui this chapter earnings and impact estimates are expressed in nominal dollars. The follow- 
up period varied across individuals, beginning as early as November 1 987 and ending as late as December 1 990. 
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Average Monthly Earnings: Treatment Group and Control Group 



$1,000 T 




$0 -1 1 • 1 1 1 i » < ' — 1 ' 1 1 1 1 < 1 

0 3 6 9 12 15 18 

Months after random assignment 

Source: Estimates based on First Follow-up Survey resp>onses. 

Notes: Sample size, treatment group = 2,980, control group = 1,439. Estimates arc regression-adjusted 
to control for differences in baseline characteristics between the treatment group and control group; see 
Appendix D. 

$493 in the first follow-up month to $801 in the eighteenth/ But this information does not 
tell us what the treatment group would have earned without access to JTPA, which is what 
we must know to calculate the impact estimates we will present shortly. 

It is the estimates underlying the control group curve that provide this information. 
Average earnings in the control group ranged from $50 1 in the first follow-up month to $774 
in the eighteenth.^ These point estimates lend two critical pieces of information to the analysis. 
First, they represent our estimate of what the treatment group would have earned, on average, 
without access to JTPA — the basis for the treatment-control group comparisons of the 
experimental impact analysis. And second, they also demonstrate the importance of making 
these comparisons in evaluating employment and training programs, since they show that, 
even without access io JTPA, control group members increased their average monthly 
earnings substantially over the 18-month follow-up period. 

4. We used ordinary least squares regression procedures to increase the statistical precision of these 
estimates, as described in Appendix D. 

5. The earnings estimates shown in Exhibit 5.1 and subsequent exhibits include wages paid to JTPA 
participants in on-the-job training positions. During the 18-month foUow-up period the program reimbursed 
employers a total of about $700 per adult male OJT participant. Among all adult men in the sample, OJT 
reimbursements totaled about $ 100 per ireatniem group member over the 1 8-monlh follow-up period. 
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Enrollment Patterms OVER Time: Treatment Group 

As explained in the previous chapter, any difference in earnings between the treatment group 
and the control group may be partly affected by the treatment group's participation in the 
program. In the early months of the follow-up period, enrollment in classroom instruction 
may have actually delayed employment for some treatment group members, whereas for 
others enrollment in on-the-job training may have led to faster job placements and thus higher 
wages than their control group counterparts during those months. To interpret property the 
pattern of treatment-control group differences, we must therefore distinguish between when, 
during the follow-up period, most treatment group members were enrolled in JTPA and when 
most of them were no longer enrolled. 

Exhibit 5.2 allows us to make this distinction between the in-program period and the 
post-program period.^ Among adult men, enrollment rates of the treatment group dropped 

Exhibit 5.2 Percentage Enrolled in JTPA Monthly: Treatment Group 





100% 


< 






80% 






c 




'i 


60% 


o 








c 
o 






40% 


& 


2 




c 




o 
o 


20% 


a. 






0% 



6 9 12 

Months after random assignment 



15 



18 



Source: Unadjusted frequencies based on enrollment and tracking daU from the 16 service delivery areas 
(SDAs). 

Note: Sample size, treatment group = 3,759. 



6 Exhibit 5.2 is based on daU from SDA records that somewhat ovcrsutc the number of persons still in 
the program at any given time because the data arc missing some tcnnina..on dates. Thus, this graph serves 
as an upper bound on the pcrccnugc of the treatment group still in the program in any given month. The extent 
to which the graph ovcrsUtes the actual enrollment .<5S is higher in the later months. 
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from 55 percent in the first follow-up month to only 5 percent in the eighteenth/ We are using 
the first month in which less than 15 percent of the treatment group was enrolled as the 
beginning of the post-program period, which was in month 7 for adult men. Thus, b>* referring 
back to Exhibit 5 . 1 , we can see thatthe positive treatment-conuol group difference in earnings 
began before most treatment group members left the program (month 3) and persisted well 
after most of those who were enrolled had left the program (months 7 through 18). 



Impacts ok Earnings: JTPA Assignees and Enrollees 

Exhibit 5.3 displays estimates of the average program impacts on the earnings of adult men 
in the sample during each of the first six quarters after random assignment and over the i 8- 
month follow-up period as a whole. As was the case for adult women in Chapter 4, it is 
important to note that these estimates of program impacts for men represent the incremental 
impact of JTPA, over and above the effects of other employment and training services that 
were available to control group members. Because not all treatment group members received 
emploxTTient and training services from JTPA and because some control group members 
received employment and training services from non-JTPA providers, the service differential 
(increment) between treatment and control group members was limited. To help interpret the 
magnitudes of these impact estimates, ou r final report will compare them to the corresponding 
differentials in the costs of the employment and training services received by treatment and 
control group members. 

Column 1 of the exhibit shows the estimated average quarterly earnings of the control 
group, which rose from $1,659 in the first -Uow-up quarter to $2,242 in the sixth, for an 
average total earnings over all quarters of i;i2,306. Column 2 presents dollar estimates of 
program impacts on earnings per JTPA assignee (treatment group member), obtained by 
taking the difference between the average earnings of treatment group members and the 
average earnings of control group members.* Column 3 expresses these dollar estimates as 
a percentage of the control group mean for the corresponding time period.^ These estimated 
inipacts reflect how much more the treatment group ca/ned as a result of its members having 
access to the program. 



7. Although 61 percent of the adult male ircalmenl group was enrolled in JTPA Title D-A at some point 
during the follow-up period, only 55 percent was enrolled in the first follow-up month, because some treatment 
group members were enrolled later. 

8. To increase the statistical precision of these estimates, we used ordinary least squares regressions. This 
reduced the standard errors of the impact estimates but did not appreciably affect the point estimates, because 
the average values of the independent variables (mainly the baseline characteristics of the treatment and control 
groups) were virtually the same for the two groups. See Appendix D for a full description of these procedures 

9. For simplicity's sake, we do not report significance levels for the percentage impacts because they arc 
the same as those in the corresponding rows of column 2. 
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Exhibit 5.3 


Impacts on Earnings: 
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Notes: Sample size, assignees « 2,980; conUol group « 1,439, Estimates are regression-adjusted to control 
for differences in baseline characteristics between the ucatment group and conUol group; sec Appendix U. 
• Statistically significant at the .10 level, »• at the .05 Icvd, at the .01 level (two-tailed test). 

Significance levels for column 3 are identical to those in column 2. Te«u of statistical significance were 

not performed for column 4. 

The estimated impacts varied from a low of $17 in the first quarter to a high of S138 in 
the third. In percentages .ms this range was from 1. 1 percent to 6.7 percent. For the follow- 
up period as a whole, the estimated impact on earnings per assignee was $550, or 4 .5 percent. 

As shown, the only impact estimates that were statistically significant at conventional 
levels were those for Ae second and third follow-up quarters. Hence, it is only for those 
quarters that we can be confident that the estimate represents a true impact, instead of a chance 
difference resulting from random sampling error. Nevertheless, two of the other quarterly 
earnings estimates in Exhibit 5.3 were statistically significant at very near conventional levels, 
as was that for the foilow-up period as a whole.'" Since the estimated impacts on employment 
for the target group as a whole, presented later in this chapter, were highly significant, we 
will at that point offer further interpretation of the earnings estimates presented here. 

Impact estimates per JTPA assignee represent the effect of having access to Title II-A. 
But not all treatment group members actually enrolled, in the program, and thus we calculated 
separate estimates of impacts per JTPA enrollee. As discussed in Chapter 2, however, we 
cannot provide a fully reliable estimate of the impact per JTPA enrollee. We can, however, 
infer the impact per enrollee by adjusting the estimated impact per assignee to account for 
the facts that 38.7 percent of all adult male assignees did not become enrolled in JTPA and 
2.2 percent of adult male control group members did enroll (despite the experiment's embargo 
on their participation)." As in the preceding chapter, we view these inferred impacts /^er 



10. Estimates for the fifth and sixth quarters, and for the follow-up period as a whole, were statistically 
significant fit the .15 level (two-tailed test) 

1 1 Appendix D details the procedures used for these adjustments. 
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enrollee as estimates of the upper bound on the average program impact on enrollees, and 
we view the estimated impacts per assignee as estimates of the lower bound on the average 
impact on enrollees.^- 

Column 4 in Exhibit 5.3 displays our inferred estimates of impacts on earnings per 
enrollee.^^ For the third quarter after random assignment (the quarter that exhibited the 
largest impact per assignee), the estimated impact per en»*ollee was $235 . Thus, for the third 
quarter our best estimate of the average impact on enroUees is in the range between $ 1 3 8 (the 
estimated impact per assignee) and $235 (the inferred impact per enrollee). 

Recall from Chapter 4 that these inferences arc based on the assumption that JTPA had 
no impact on those assignees who did not enroll in the program. If these nonenrollees 
received little or no JTPA service, and thus operienced a negligible JTPA impact, the true 
impact on enrollees probably closer to $235 in the third quarter.'^ But if nonenrollees 
did receive substantial JTPA services, and thus experienced impacts similar to those that the 
enrollees experienced, the true impact on enrollees was probably closer to $138 in that 
quarter. Without complete information on the JTPA services received by nonenroUed 
treatment group members, we cannot determine where in this range the answer lies . To reflect 
this uncertainty, we did not attempt to calculate the statistical significance of the inferred 
impacts per enrollee, and in the remainder of this chapter we focus primarily on impacts per 
assignee. 



Impacts on the Distribution of Earnings: JTPA Assignees 

The preceding findings represent the average program impacts per assignee or per enrollee. 
Among individual men in the sample some may have experienced larger impacts, while others 
experienced smaller, negligible, or even negative impacts. To better understand how JTPA 
works, we would also like to know what this distribution of impacts looks like. For example, 
does the $138 average impact per assignee in the third quarter represent a large positive 
impact on a few sample members or a modest positive impact on many? 



12. It is possible thai the program impact on noncnrollcd ircatmcnt group members was the opposite of 
any impact on the enrollees. If this were so, vvhich seems unlikely, the estimated impacts per enrollee would 
underrtate the magnitude of the average impact on enrollees. 

1 3. As explained in Appendix D» the adjustment factor used to derive impacts per enrollee from impacts 
per assignee is \f(r-c), where r is the enrollment rate (the proportion of treatment group members who were 
enrolled in JTPA) and c is the crossover rate (the proportion of control group members who were enrolled in 
JTPA). Since these two rates are fixed for any given group or subgroup, the ratio of impacts per enrollee to 
impacts per assignee is also fixed for any given group or subgroup. Tiius, for example, for the adult male target 
group, the impact per enrollee is 1.71 times the impact per assignee for all outcomes in all time periods. 

14. Appendix F explores the evidence of JTPA services received by treatment group nonenrollees. 
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As explained in Chapter 4, we cannot answer this question because we cannot directl\ 
observe the distribution of program impacts on the earnings of individual treatment group 
members. We can, however, observe average program impacts on the earnings distribution 
of the treatment group. Here the question is how JTPA changed the shape of the earnings 
distribution . For example, did the program reduce the percaitagc of treatment group members 
with low earnings, or did it increase the percentage with higher earnings? 

Exhibit 5.4 addresses these questions by comparing the distributions cf total 18-month 
earnings for JTPA assignees (the treatment group) in column 1 and for the control group in 
column 2. As in Chapter 4, the categories of earnings shown on the left side of the exhibit 
are defined so as to divide the earnings distribution of the control group into those with zero 
earnings and quartiles of those with positive earnings.'^ Column 3 shows the difference 
between the two groups for each of the five earnings categories. These differences represent 
the impacts of JTPA on the earnings distribution. 



Exhibit 5.4 Impaas on the Distribution of Total IS-Month Earnings: JTPA Assignees 
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Notes: Sample size, assignees « 2,980; conirol group « 1,439. For the estimation procedure, see Appendix D. 
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t-test). 



The zero earnings category exhibits the largest treatment-control group difference, and 
the only one that is statistically significant. This finding indicates that JTPA reduced the 
percentage of assignees with zero earnings by -2.4 percentage points. In other words, the 
program increased the percentage of assignees who became employed, which i.T consistent 
with the estimated impacts on employment, that wc will turn to next. Because the treatment 
group and control group percentages were quite similar for the other four categories, our test 
of the overall impact on the earnings distribution was not statistically significant. 



15. The dislribuiion of control group members for those with positive camings docs not equal exact 
quartiles because the camings categories used to dcfmc this distribution were specified to the nearest $100. 
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Impacts on Employment: Adult Men Overall 

This section examines the impacts of JTPA Title II-A on the emploNinent of adult men. We 
report estimated impacts using three measures of emplo\Tnent — the percentage employed. 
the average number of weeks worked, and the average number of hours worked — for each 
follow-up quarter and for the 18-month period as a whole. All three measures include zeros 
for sample members who were not employed. 

Exhibit 5.5 displays the three measures in separate panels. Note first that the top panel 
indicates that 59.4 percent of the control group was employed at some time during the first 
quarter after random assignment. The employment rate of the control group grew continualK' 
over the remaining follow-up quarters, reaching 69.2 percent in the last one. During the 18 
months as a whole, 83.6 percent of the control group was employed at some time.'^ 

Nevertheless, the estimated impacts per assignee on the percentage employed were 
statisticalh' significant for the second, third, and fourth quarters and for the follow-up period 
as a whole, ranging in these quarters from an estimated 3.4 to 5.2 percentage points (column 
2). The inferred impacts per enrollee ranged from an estimated 5.7 to 8.9 percentage points 
in those quarters (column 4). Overall, the program increased the proportion of the treatment 
group ever employed during the 18-month period by an estimated 2.8 percentage points for 
assignees (which was statistically significant) and by an estimated 4.8 percentage points for 
enrollees. Note, however, that the impact on employment rates appeared to decline toward 
the end of the follow-up period. 

The middle panel in the exhibit shows estimates of program impacts on the number of 
weeks worked by sample members. This measure is an average for all adult men in the sample, 
and thus it includes zeros for men who were not employed. The average number of weeks 
worked by control group members rose continually over the period, which again indicates that 
the labor market prospects of treatment group members would have improved to some extent 
even without access to JTPA. But JTPA had a statistically significant impact on weeks 
worked in everj^ follow-up quarter except the first, when many treatment group members were 
still in the program. The estimated impacts per assignee ranged from 0.3 to 0.5 weeks, or 
4.4 percent to 6.7 percent. The inferred impacts were larger by a factor of 1 .7 1 for enrollees. 
Over all quarters the program increased the number of weeks worked by assignees by 2.2 
weeks, or 4.8 percent, which was statistically significant. 



16. Because the oulcome shown in the top panel of Exhibit 5.5 is the percentage of ihe sample ever 
employed in the relevant time period (quarter or 18-month follow-up period), quarterly proportions do not sum 
or average to the proportion for the follow-up period as a whole. 

17. Theratioof impacts per enrollee to impacts per assignee may difier somewhat from 1.71 because of 
rounding. 
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Exhibit X5 Impacts on the Percentage Employed and on the Mean Number of 
Weeks and Hours Worked: JTFA Assignees atid Enrollees 
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The bottom panel in Exhibit 5.5 shows the estimated impacts on the average number of 
hours worked, which again includes zeros for men who were not employed. The average for 
control group members rose until the fourth quarter but then remained about the same. 
Average hours worked among assignees, on the other hand, rose by even more during the 
period, and program impacts were statistically significant in three of the last four quarters. 
Over the full follow-up period the estimated impact per assignee was also statistically 
significant at an additional 84 hours, while the inferred impact per enrollee was 143 hours. 

Thus, the estimated impacts of JTPA on the overall employment of men were statistically 
significant in most quarters and for the follow-up period as a whole. If, as indicated in the 
next section, the program did not reduce the average hourly earnings of those men who were 
employed, this increase in employment implies a corresponding increase in earnings. 
Together with the fact that the estimated impacts on earnings presented in the previous section 
were statistically significant at close to conventional levels, this evidence suggests that the 
program did increase average earnings for men. 

Impacts on the Components of Earnings: Adult Men Overall 

In this section we attempt to distinguish between the effects on the average number of weeks 
and hours worked per Ji^signee that simply reflect higher employment rates among all 
assignees and those that reflect additional weeks and hours worked among only those men 
who were employed. As in the previous chapter, we do this by shifting from average weeks 
and hours worked by all assignees (including those who did not work) to average weeks and 
hours per week worked by only those assignees who did work. This analysis is based on 
the fact that average earnings per assignee can be decomposed as follows: 

earnings workers ^ weeks ^ hours ^ earnings 

assignee assignee worker week hour 

Each of the four components of earnings in this relationship reflects a different aspect 
of labor market success. Workers per assignee reflects the ability of assignees to find jobs — 
the "pure" employment effect. Weeks worked per worker reflects both how quickly 
assignees found jobs and how long they held them, or the stability of their emploj-ment. 
Hours worked per week worked reflects the mix of workers' part-time, full-time, and 
overtime work, that is, whether they were more likely to find a full-time job. And earnings 
per hour worked reflects what workers were paid for the time they worked. Appendix D 
explains how we estimated program impacts on each of these earnings components, and 
Exhibit 5.6 presents our findings. 
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Exhibis 5. 6 Percentage Impacts on Earnings and Its Componenis: JTPA Assignees 



Period 



Quarter 1 
2 
3 
4 
5 
6 

All quarters 



Earnings 
per assignee 
0) 


Workers 

per 
assignee 
(2) 


Weeks 
worked 
per worker 
(S) 


Hours worked 
per week 
worked 
(4) 


Earnings 
per hour 
worked 
(5) 


1.1% 


1.9% 


-2.3% 


-0.4% 


2.0% 


6.3 


6.4 


-1.1 


-0.6 


1.6 


6.7 


8.0 


U.2 


0.0 


0.0 


3.1 


5.0 


-0.6 


-1.2 


-0.1 


4.7 


2.4 


3.1 


0.8 


-1.6 


4.6 


2.8 


3.4 


-0.1 


-1.5 


4.5 


3.4 


1.4 


-0.3 


0.0 



Source: Efiioutci based on First Follow-up Survey TopoMC*. .... , , 

Noi«: Sample size. Kiixoees- 2,980; control jroup « 1,439. EMimau* aw «»re«*on^ju«i^io control for 
differences in baseline chatacterittici between Ibe ueaunent jroup and control jroup; tec Appeodix D . 
Columns 2 thfoujb 5 dispUy the impact as a pocenuje of the correipondinx coouol mean (not shown). For 
column 2 this means the impact on the employment rale ii diiplayed as a perocniaje of ihc mean rate for the 
conirol jroup. Tesu of satiflical lixnificance were ooc performed for any of the columns m this exhibit. 

Column 1 in the exhibit repeats the estimates oi percentage impacts on earnings per 
JTPA assignee given earlier in Exhibit 5.3. The remaining four columns present estimates 
of the percentage impact on each earnings component. Because the percentage impacts on 
the four components in each row sum to approximately equal the percentage impact on 
earnings in the same row, they indicate the relative contribution that each component made 
to the impact on earnings. No significance levels are shown in Exhibit 5.6. The significance 
levels of the impacts on total earnings (column 1) and workers per assignee (column 2) are 
the same as those in Exhibits 5.3 (for earnings) and 5.5 (for percentage employed). Because 
the last three columns were estimated indirectly, no tests of significance were calculated for 
the estimated impacts on these outcomes.^* 

The results indicate that the positive program impact on the earnings of men reflects an 
increase in the amount of time they worked, not an increase in the average amount they were 
paid for the time they worked. This conclusion holds for all six follow-up quarters, but it is 
most obvious in the row for the full follow-up period, where the 4.5 percent impact on total 
earnings is manifested almost exclusively in the 3.4 percent increase in the proportion 
en-p!oyc-d (workers per assignee) and the 1 .4 percent increase in weeks worked per worker. 
O /erail there was no change in earnings per hour worked and virtually no change in hours 
worked per week worked. 



18. See Appendix D for an explanation of the procedures used to estimate impacts on these outcomes. 
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It is important to recognize that the finding of no effect on hourly earnings applies to the 
overall average for all workers in the treatment group. Program-induced changes m the 
composition of the group who worked may mask impacts on the average hourly earnings of 
subgroups of workers. For example, the program may have increased the hourly earnings of 
those who would have worked anyway but enabled more low-wage workers to become 
employed. The latter impact could have offset the former in its effect on the overall average 
hourly earnings of workers. Thus, the impact findings in Exhibit 5.6 do not necessarily reflect 
the impacts of JTPA Title II-A programs on the productivity of adult men. 

Appendix G describes a separate, nonexperimental analysis of "latent wage rates'' that 
we conducted to explore this issue further. Estimates of latent wage rates are designed to 
measurethe productivity of individuals, including bodithosewho were employed (whosewagc 
rates can be observed) and those who were not employed (whose wage rates cannot be 
observed). Findings from this analysis suggest that access to JTPA had very little effect on 
the latent wage rates of adult male assignees in the sample. Tney therefore support the 
interpretation of a pure employment effect. For reasons discussed in the appendix, however, 
the results of the latent wage rate analysis may not be reliable. 



Impacts on Earnings, Employment, and Eamings Components: 
Adult Men Recommended for Each Service Strategy 

The preceding impact estimates, for all adult men in the 1 8-month study sample, were averages 
for many different types of men recommended by program staff for a variety of prog'-am 
services. This section focuses more narrowly on the subgroups of adult men recommended 
for each of the three clusters of services, or service strategies, defined in Chapter 2: the 
classroom training, OJT/JSA, and other services strategies. 

As described in Chapter 2, the service strategies were defined in terms of the JTPA 
services recommended for sample members before they were randomly assigned to treatment 
or control status. Using recommended services, instead of services actually received, as the 
basis for this definition was necessary to allow us to identify the control group counterparts 
for the treatment group members in each service strategy subgroup. 

Because recommended services were a good predictor of services received, this approach 
produced service strategN' subgroups that did indeed receive distinctly different JTPA services. 
But recommended services were not a perfect predictor of services received, and more than 
one service was recommended for many sample members. Each service strategy subgroup 
in fact received a mix of services, not a single isolated service. Hence, the impact findings 
reported for each service strategy subgroup reflect the effects of the mix of services each 
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subgroup received, not the impact of a specific program semce. More specificalK', as 
indicated earlier, our impact estimates reflect the incremental effect of the difference between 
the mix of services received by the treatment group and the mix received by the control group. 

In addition, it is important to note that the kinds of men recommended for each service 
strategy differed. Thus, any differences in program impacts across service strategy subgroups 
cannot be attributed solely to differences in the kinds of services received. The impact findings 
in this section must therefore be viewed separately— as indicators of how well each strategy 
worked for the distinct group of people it actually served. Differences in unpacts across 
service strategies do not indicate what would happen if a particular group were shifted from 
one service strategy to another. 

We first describe how the employment and training services received by men 
varied across the three service strategy subgroups. We then examine the differences in 
services received between the treatment group and conuol group within each subgroup. The 
cross-strategy comparisons refer back to the comparisons in Chapter 3 showing how the 
subgroups differed on this dimension, while the treatment-control group comparisons reflect 
the increment in services attributable to JTPA Title II-A. It is this latter difference in service 
receipt between the treatment and control groups that is the source of program impacts for 
each service strategy subgroup. As part of this discussion, we also present estimates of 
program impacts on the attainment of a high school diploma or GED certificate associated 
with participation in school or training. 

We then examine how the three service strategy subgroups differed from one another in 
terms of their baseline characteristics. The remainder of the section presents the estimated 
program impacts on each subgroup, based on the same measures of earnings, employment, 
and earnings components as those in the earlier sections on adult men overall. 



Differences in Employment and Training Services Received: Treatment 
AND Control Groups within Each Service Strategy Subgroup 

Recall from Exhibit 3.12 in Chapter 3 that 71.2 percent of the adult male treatment group 
members recommended for the classroom training service strategy became enrolled in JTPA 
Title II-A, whereas 56.6 percent of those recommended for the OJT/JSA strategy and 58.9 
percent of those recommended for the other services strategy became enrolled in the program. 
Thus, the majority of adult male treatment group members enrolled in JTPA (60 .8 percent 
overall), but the enrollment rates varied substantially between the classroom training 
subgroup and the other two service strategy subgroups. 

Moreover, the three service strategy subgroups differed substantially in the specific JTPA 
services they Ultimately received. Specifically, as Exhibit 3.19 indicated, 85.:i percent of the 
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adult male enrol lees recommended for the classroom training strateg\' received one or both 
of two JTPA services: classroom training in occupational skills and basic education. In the 
OJT/JSA subgroup, 86.5 percent of enrollees received on-the-job training, job search 
assistance, or both, and 88.7 percent of those enrollees recommended for the other services 
strategy received job search assistance, miscellaneous services, or both. 

Thus, in terms of the services received by treatment group members, each service strategs' 
rests mainly on two specific JTPA services, the defining service for that strateg\' and a 
secondar>' service. To the extent that treatment group members received the defining service 
for their strategy, the result reflects the primary service recommendation of SDA staff. 

But the impact of JTPA depends on the difference between the services treatment group 
members received after being allowed to enroll in JTPA and the services they would have 
received if they had been excluded from the program. As discussed in Chapter 4, this 
difference is represented by the difference in services received, on average, between treatment 
group members and control group members. 

Exhibit 5.7 displays estimates of this service difference for men who were recommended 
for each service strateg>'. The exhibit compares both the perceintage of treatment group 
members and control group members who received each service (their likelihood of receiving 
the service) and the average number of hours of each service received (the amount of the 
service they received). 

It is important le^note that, for reasons discussed in Chapter 4, estimates for different 
services were obtained from different data sources and are subject to different limitations. 
Estimates of the receipt of classroom training in occupational skills and basic education, 
for example, include services from both JTPA and non-JTPA providers, measured fi-om 
responses to the First Follow-up Survey. The estimates for these two services therefore offer 
a reasonably complete treatment-control group comparison.*^ 

Estimates of the receipt of on-the-job training znd work experience, on the other hand, 
apply to JTPA Title II-A services only and were obtained from SDA enrollment and tracking 
data. Because these services are typically provided only through JTPA, the estimates for each 
offer a fairly reliable treatment-control group comparison. 

Job search assistance and miscellaneous services, however, are typically available 
from non-JTPA providers and thus are accessible to control group members. Because SDA 



1 9. The survey-based estimates of receipt of classrcwm training in occupational skills and basi" education 
are somewhat inconsistent with the SDA data on service receipt shown in Chapter 3, and they may in fact 
understate the incidence of these services. An analysis of these inconsistencies will be presented in our 
forthcoming final report. 
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Exliibit 5. 7 Receipt of Employment and Training Services: Treatment Croup and Control Group, 
by Service Strategy Subgroup 



Specific program 
service 



Classroom training in 

occupational skills 
Basic education 
On-lhc-job training * 

(JTPA only) 
Work experience * 

(JTPA only) 
Job search assistance * 

(JTPA only) 
Miscellaneous * 

(JTPA only) 



■ Mean hours of service 

Percentage receiving service per sample member 

Treatment Control Difference, Treatment Control Difference, 
group group in % pis. group group in hours 
^nf n!^ (3) (4) (5) (6) 



Classroom training subgroup 



Classroom training in 

occupational skills ^ * 40.1% 24,2% 16.0% 235 

Basic education'* 10.0 4.9 5.1 38 
On-the-job training 

(JTPA only) 5.4 0.2 5.2 38' 

Work experience * * ^ 

(JTPA only) 1.7 0.0 1.7 8 

Job search assistance* 

(JTPA only) 12.4 

Miscellaneous''* 

(JTPA only) 9.7 ^ " 



OJT/JSA subgroup 



9.2% 
3.9 

26.6 

2.4 

30.2 

6.8 



9.2% 
4.1 

0.5 

0.0 



0.0% 
-0.3 

26.1 
2.4 



58 
11 

119' 

lo' 



140 
29 

< 

1 
0 



55 
13 

5' 

c 

0 



95 
9 

37 
8 



3 
-2 

114' 

10' 



(Continued) 

administrative records were our only reliable source of data on receipt of these services, we 
were unable to produce estimates for control group members. Exhibit 5.7 therefore does not 
allow for treatment-control group comparisons for these two categories of services. 

The average number of hours of service receipt in columns 4 through 6 include the 
experience of both sample members who received the service and those who did not. To 
calculate an estimate of the average hours of receipt for only those sample members who 
actually received a service (service recipients), simply divide the estimate in column 4 or 5 
(hours for all treatment group or control group members in the subg .oup) by the estimate in 
column 1 or 2 (divided by 100). 
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Exhibil 5. 7 Receipt of Employment and Training Services: Treatment Group and Control Group, 
Service Strategy Subgroup (continued) 



Specific program 
service 


Percentag 


e receiving 


service 


Mean hours of service 
per sample member 


Treatment 
group 

0) 


Control 
group 
(2) 


Difference , 
in % p:s. 
(3) 


Treat inent 
group 
C4) 


Control Difference , 
group in hours 

(5) (6) 








Other services Siibgroup 




Classroom training in 












occupational skills 


11.6% 


9.9% 


1.7% 


73 


77 -4 


Basic education ^ * 


5.7 


4.5 


1.2 


18 


10 8 


On-the-job training* * 












(JTPA only) 


4.9 


0.4 


4.5 


29' 


2 27 


Work experience* * 








c 


0^ 6^ 


(JTPA only) 


0.9 


0.0 


0.9 


6 




Job search assistance * * 












(JTPA only) 


25.8 










Miscellaneous * * 












(JTPA only) 


29.0 











Sources: 

♦ UMdjuiled frcqucocici ia this row are b**e<J oo Fin* FoUowmp Survey dau oo receipt of Ihc •crvice from any provider. 
^ * Unadjusted frequcaciea in ihit row are based oo corotlmcnt aod uackioc dau from the 16 SDAs, the best avaiUble dau 

OQ receipt of this service. Altboufh the du are for JTPA Title H-A-fuoded services only, this service is typically noi 

funded by ooo-JTPA providers. 
« ♦ ♦ Unadjusted frtqucacies in ihis row aie alto based oo SDA cnroUmcat and tn^^ dau. Comparable dau oo receipt of 

this service from other providers were aoi available; nor were comparsble dau on receipt by control group members. 
Noics: Sample sixe, classroom tfainioc nibg roup: treatmjot group «* 732, control group 325; OTT/JSA subgroup: treaunent 
group ■ 1.516. control giw^ « 734; other terviccs subgroup: Ireaimcot group « 732. control group » 380. Because of missing 
dau. sample sires for services calculated from differeot dau sources may vary. Tests of Matistical significance were not 
performed for this exhibit. 

a. Lasting loQger than ooe week. 

b. Lasting longer than one week. *Basic educatioo* includes Adult Basic Education (ABE), high school or General 
Educational Development (GED) preparation, snd English as a Second Language (ESL). 

r. Hours, assuming a full-time job at 40 hours per week. 

d. "Miscellmeous* includes ssseunKnt, job-readiness training, customized training, vocational exploration, job shadowing, 
and tfyout employment, among other services. 



Before turning to the findings in Exhibit 5.7, note that one cannot sum the percentages 
receiving services in each column because individual sample members might have received 
more than one service. 

The Classroom Training Subgroup. As shown in the first panel of the exhibit, although 
all men in this subgroup were recommended for classroom training in occupational skills, only 
40.1 percent of the treatment group reported receiving this service. The principal reason why 
some treatment group members did not receive this service was that 28.8 percent of those in 
the classroom training subgroup never enrolled in JTPA (Exhibit 3.12 in Chapter 3). As 
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expected, a number of control group members received instructional services, because non- 
JTPA funding was readily available for these services and because, as demonstrated by their 
application to JTPA, control group members were motivated to seek out these resources. 

JTPA was therefore not the "only game in town" for instructional services . Nevertheless, 
the estimates in the first panel of the exhibit show that the program was an incremental source 
of those sen'ices. In the classroom training subgroup, treatment group members were more 
likelv than control group members to receive classroom training in occupational skills (40. 1 
percent of the treatment group versus 24.2 percent) as well as more likely to receive basic 
education (10.0 percent versus 4.9 percent).^' The treatment group v.-as also more likely than 
the control group to receive on-the-job training and work experience. 

The last three columns in the first panel indicate that the average differences in the 
amounts of sen/ices received between all members of the treatment group and the control 
group were 95 additional hours of classroom training in occupational skills and 9 additional 
hours of basic education— additional hours of service receipt attributable to the treatment 
group's having access to JTPA. These differences were mainly a consequence of the 
difference in the likelihood of receiving each service, and not primarily a consequence of a 
difference in the amount of the service received per service recipient. 

To see this, licts that among service recipients only, the average hours of receipt of 
classroom occupational skills training were very similar for the treatment group, at 586, and 
the control group, at 579 hours.^' Moreover, although we have seen that the treatment group 
overall was more likely than the control group to receive basic education, control group 
recipients actually received a greater amount of basic education, at 592 hours, than treatment 
group recipients of this service, at 380 hours, on average. The content and quality of 
Instruction may, of course, have differed between treatment and control groups. 

Finally, 12.5 percent of the treatment group in the classroom training subgroup received 
job search assistance, and 9.7 percent received miscellaneous services, the last two categories 
in the panel. 

TheOJT/JSA Subgroup. In the OJT/JS A subgroup 27.1 percent of the treatment group 
received on-the-job training, and 30.7 percent received job search assistance, as shown in the 
second panel of Exhibit 5.7. Again, the main rea-Jon why some treatment group members did 
not receive the defining service for the strategy— OJT— was that 43 .4 percent of the treatment 



20. As previously noted, these survey-based estimates may undersute the numbers of men who received 
classroom training and basic education. 

21 These numbers are derived bv dividing the average number of hours of service receipt per treatmenl 
or control group member (column 4 or 5) by the coiresponding proportion receiving the service (column 1 or 
2, divided by 100). 
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group in this subgroup were never enrolled in JTPA during the follow-up period (Exhibit 3.12 
in Chapter 3). 

As expected, however, the likelihood of receiving on-the-job training was much higher 
for men in the OJT/JSA subgroup than for those in the other two service strategy subgroups. 
The likelihood of receiving job search assistance was also much higher for this subgroup than 
for the classroom training subgroup, but it was similar to the rate of receipt for the other 
services subgroup. 

Regarding treatment-control group differences, recall that we do not ha^ e data on receipt 
of on-the-job training from non-JTPA providers; but since few of those providers offer OJT, 
the estimates of treatment-control group differences in OJT receipt from JTPA pro\'iders are 
probably reasonably reliable. If that is the case, the estimates in the second panel of Exhibit 
5.7 suggest that in the OjT/JSA subgroup, treatment group members were more likely than 
control group members to receive on-the-job training (27.1 percent versus 0.5 percent). 

Column 6 also indicates that JTPA provided an additional 1 14 hours of OJT, averaged 
over oil treatment group members. But among service recipients only, the average number 
of hours of OJT receipt was higher for the control group than for the treatment group Thu s, 
similar to the case for adult men in the classroom training subgroup, the positive tr-atment- 
control group difference in hours of OJT receipt shown in column 6 was primarily the 
consequence of the difference in the likelihood of receiving OJT, not of the difference in the 
amount of the service received by those who received it. 

In the OJT/JSA subgroup, treatment-control group differences in the receipt of classroom 
training in occupational skills and basic education were negligible, in terms of both the 
likelihood and the amount of the service received overall, and in terms of the average number 
of hours of the service received by service recipients only. As noted above, data limitations 
precluded our calculating estimates of differences in the receipt of the last two categories, job 
search assistance and miscellaneous services. 

The Other Services Subgroup. Because the same data limitations apply to the other 
services subgroup, it was not possible to calculate treatment-control group comparisons for 
these last two categories of services. We do know from Chapter 3, however, that 58.9 percent 
of adult male treatment group members recommended for the other services subgroup 
subsequently enrolled in JTPA (Exhibit 3.12) and that 88.7 percent of those who enrolled 
received job search assistance, miscellaneous services, or both (Exhibit 3.19). 



22, Among service recipients only, the average number of hours of service receipt was 439(119 divided 
by 27.1/100) for the treatment group. It was 875 for the control group, based on the more precise estimates 
(4.74 hours divided by 0.542/100) underlying the rounded estimates in the exhibit. 
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As shouTi in the last panel of Exhibit 5.7. 25.8 percent of adult male treatment group 
members received job search assistance, and 29.0 percent received miscellaneous ser^'lces. 
These were also the two most common services received by adult female treatment group 
members in the other services subgroup. 

The only information available to assess treatment-control group differences in service 
receipt for the other services subgroup is that on the less common services received by this 
subgroup. As shoxvn in the first four rows of the last panel, treatment group members were 
slightly more likely than control group members to receive classroom trainmg m occupational 
skills, basic education, and work experience— and somewhat more likely than control group 
members to receive on-the-job training. 

Nevertheless, because we cannot measure control group members' receipt of the two most 
common services received by adults in this service strategy subgroup— job search assistance 
and miscellaneous services— we cannot reliably measure the overall treatment-control group 
difference in emplojment and training services for members of the other services subgroup. 

Summary of Differences in Service Receipt. The preceding comparisons of 
service receipt across service strategy subgroups and between the treatment and 
control groups within each subgroup can be summarized as follows: 

• Many men in the treatment group did not receive the & ^rvice recommended for 
them, either because they were never enrolled in JTPA or because they received 
a service other than the primary one recommended for them. 

. Nevertheless, the three ser/ice strategy subgroups, which were formed on the 
basis of service recommendations by JTPA intake staff, do appear to represent 
distinctly different clusters of services actually received and are therefore useful 
for analyzing the effects of different service strategies. 

• Within each service strategy subgroup most men who enrolled in JTPA received 
one or both of two key services, which varied by service strategy: In the 
classroom training subgroup these wert classroom training in occupational skills 
and basic education; in the OJT/JS A subgroup they were on-the-job training and 
job search assistance; and in the other services subgroup they were job search 
assistance and miscellaneous services. In all three subgroups the main reason 
why a portion of the treatment group did not receive one of the two key services 
in the strategy was failure to enroll in JTPA. 
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• The services actually received bN' each service strateg>' subgroup were consistent 
with the recommendations of intake suff to a considerable extent, but the\ 
differed in some important ways. Most important was the finding that only 27. 1 
percent of adult male treatment group members recommended for OJT (the 
OJT/JSA service strateg\') actually received that service. Because 30.7 percent 
of adult male treatment grou p members recommended for OJT received j ob search 
assistance, it is most appropriate to characterize the service strateg>' as one based 
on placement in employment, with or without subsidized training. 

Treatment-control group differences in the average amount of service received 
were relatively modest, where those differentials could be measured. The average 
amount of additional occupational skills training rece ved by treatment group 
members in the classroom training subgroup was onl> 95 hours. Similariy, the 
treatment-control group difference in receipt of OJT by men in the OJT/JSA 
subgroup was only 1 14 hours. These modest service differentials reflect the fact 
that not all treatment group members received these services and, in the case of 
occupational skills training, that some control group members received the 
service. 

It is therefore important to note that the overall magnitude of the program impact does 
not reflect the effect of JTPA services versus no employment and training services. Rather 
it reflects the effect of the increment in services received by treatment group members beyond 
what they would have received without JTPA. Furthermore, when comparing the magnitudes 
of two service strategy subgroups estimated impacts, bear in mind that they do not represent 
impacts of two specific program services, but rather impacts of two different service mixes. 

The costs of thesedifferentservicemixesmay vary considerably. And most importantly, 
the treatment-control group differential in service costs may vary subsuntially across the 
service strateg>' subgroups. Thus, the differences in impacts across service strategy subgroups 
ihat we report later in this section may not fully correspond to their differences in cost- 
effectiveness. As mentioned earlier, our forthcoming final report will present a benefit-cost 
analysis that takes these cost differences into account. 

Impacts on High School Attainment. A number of men in the sample achieved a high 
school diploma or GED certificate as part of the school or training services described above. 
We refer to this result as the attainment of a training-related high school credential. Exhibit 
5.8 shows the percentage of treatment group and control group members who reported both 
participating in school or training and attaining a high school diploma or GED certificate at 
some time during the 18-month follow-up period. The treatment-control group difference in 
this outcome represents the additional educational attainment attributable to having access to 
JTPA. 
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Estimates of this impact are presented for the target group of adult men as a whole and 
for high school dropouts only within each service :;trateg\' subgroup. Focusing onl\' on high 
school dropouts isolates the impact of the program on those sample members for whom this 
was a potential impact. The estimated impact for the entire treatment group, on the other hand, 
indicates the relative importance of this outcome for the group as a whole. 

As shown in the last panel in Exhibit 5.8, overaU 3,8 percent of adult male assignees and 
2.0 percent of adult male control group members attained a training-related high school 
credential. The statistically significant 1 .9 percentage point difference represents the impact 
of JTPA on this outcome for all JTPA assignees, .\mong high school dropouts only, 12.7 
percent of the assignees and 6.7 percent of the control group attained a training-related high 
school credential. Hence, JTPA had a statistically significant impact of 6.0 percentage points 
on adult male high school dropouts. 

But these overall findings for men mask important variations among the three service 
strategy subgroups. The most striking findings are for classroom training, the strategy that 
involved the most basic education (sec Exhibit 5 .7) and that was therefore the most likely to 
lead to attainment of a training-related high school credential. Of the high school dropouts 
in the classroom training subgroup, 27.3 percent of the treatment group attained a training- 
related high school credential, whereas only 11. 3 percent of the control group did. Hence, 
JTPA produced a statistically significant 16.0 percentage point increase in the likelihood that 
the high school dropouts would attain a training-related high school credential. But, because 
high school dropouts comprised less than a quarter of the men in the classroom training 
subgroup (see column 1), the impact of JTPA for assignees overall was only a 4.5 percentage 
point Licrease. 

The OJT/JS A subgroup and the other services subgroup both received less basic 
education than the classroom training subgroup, and both had a ver>' small treatment-control 
group difference in the receipt of basic education (Exhibit 5.7). As a result, one would not 
expect these service strategies to affect appreciably the attainment of a high school credential. 

As expected, the impact on the attainment of a training-related high school credential was 
negligible and not statistically significant for the other services strategy. But JTPA produced 
a statistically significant 4.0 percentage point increase for high school dropouts in the 
OJT/JSA subgroup, which translates into a 1.2 percentage point increase for all assignees. 
Thus, even though the impact was statistically significant, it was small in magnitude and 
therefore not necessarily inconsistent with the fact that treatment and control group members 
in the OJT/JSA service strategy were about equally likely to receive basic education. 
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Exliibit 5.8 Impacis on Aiiainmeni of a Training-Relaud High School Diploma or 
GEO Cerxificate: JTPA Assignees Overall aiul High School Dropoui 
Subgroup, by Service Strategy Subgroup 



Percemage atiaining a iraining-relaied 





Sample 
0) 


Control 

Assignees group 
(2) (S) 


Difference, 
in % pts. 
(4) 






Classroom training subgroup 




Full sample 


1,018 


6.7% 2.2% 


4.5%*** 


High school dropouts 


234 


27.3 11.3 


16.0*** 






OJT/JSA subgroup 




Full sample 


2,124 


2.6 1.5 


1-2** 


High school dropouts 


683 


8.4 4.4 


4.0** 






Other senices subgroup 




Full sample 


1,044 


3.4 2.8 


0.6 


High school dropouts 


341 


10.2 8.7 


1.5 






All subgroups 




Full sample 


4,186 


3.8 2.0 


1.9*** 


High school dropouts 


K258 


12.7 6.7 


6.0*** 



Sources: UEadJusied frequencies based on Background Information Fonn responses and Fim FoHow>up 
Survey responses. 

a. Tmiment and oonitxjl {roups combined . 

•Aiuinmcni of a training- related high school credcniial" is dcfioed as the combination of having 

received some school or inining service and having atuined a high school diploma or General 

Educational Development cettificate at some limt ' .nng ihe l^month follow-up period. 
• Sutisiically significant at the .10 levej, at the .05 level, ••• at the .01 level (two-iailcd test). 

Differences in Baseline Characteristics across Service 
Strategy Subgroups: JTPA Assignees 

As explained in Chapter 2, JTPA staff recommendeu sample members for the three service 
strategies on the basis of their individual work experience and educational needs, as well as 
their personal preferences. Variations in impacts among the three service strategy subgroups 
will therefore reflect not only differences in services received by these three groups, but also 
differences in their personal characteristics. 

It is therefore important to recognize that the three service strategy subgroups differ. As 
shown in Exhibit 5.9:" 

• The classroom training subgroup was the best educated group; only 25 .2 percent 
did not have a high school diploma or GED certificate upon application to JTPA 



23. For a more detailed description of tiic baseline characteristics of these subgroups, see Bloom (1 991 ) 
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Exhibit 5.9 StUcted Baseline Characteristics.- JTPA Assignees, h- Service Strategy Subgroup 



Characierislic 



Ethnidty 

WTiile. non-Hispanic 
Biack, non-Hispanic 
Hispanic 

American Indian or Alaskan Native 
Asian or Pacific Islander 
Barriers to employment 
Receiving cash welfare 
No high school diploma or 

GED cerlificaic 
Worked less than 13 weeks 

in past 12 months 
Number of barriers 
None of the above 
One of the above 
Two of the above 
All three of the above 
Work and training histories 
Ever employed 
Mean individual earnings 

in past 12 months 
Hourly earnings in most recent job 
Never employed 
Less than $4 
$4 or more 
Employed upon application 
Previously received occupational 
training 

Public assistance status ^ 
Receiving any public assistance 
Receiving AFDC 
Receiving food stamps ^ 
Receiving other public assistance 



AU 
subgroups 
(1) 


Classroom 
training 
subgroup 

(2) 


OJT/ 
JSA 
subgroup 

\^) 


Other 
services 
subgroup 
(4) 


57.0% 


46.1% 


68.9% 


46.0% 


28.8 


39.5 


17.6 


39.5 


9.7 


8.7 


9.7 


10.7 


1.6 


1.3 


1.6 


1.4 


2.9 


5.1 


1.5 


3.4 


14.6% 


16.3% 


16.6% 


9.4% 


31.2 


25.2 


32.4 


34.6 


41.5 


39.8 


Ad A 


46.3 


21.1 


41.2 


38.0 




41.1 


38.6 


40.5 


44.8 


17.9 


17.7 


17.5 


18.7 


3.4 


2.5 


3.9 


3.1 


91.3% 


90.4% 


93.9% 


on ACL 


$3,948 


$3.9i0 


$3,994 


$3,827 


8.7% 


9.6% 


6.1% 


12.6% 


19.8 


21.5 


21.8 


14.5 


71.5 


68.9 


72.1 


72.9 


13.1 


14.8 


13.0 


11.8 


47.2 


46.7 


45.8 


50.3 


37.0% 


39.2% 


38.8% 


31.7% 


6.0 


7.8 


6.5 


3.4 


28.7 


29.3 


30.9 


24.0 


18.9 


21.7 


19.0 


16.0 



{Contit%md) 
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Exhibii 5.9 Selected Baseline Characteristics: JTPA Assignees, by Serxice Strategy Subgroup 
(continued) 



Classroom OJT/ Oih^r 
AU training JSA services 
subgroups subgroup subgroup subgroup 
Characteristic (J) Q) (Jl (^) 



AFDC histor>- 



Never AFDC case head 


91.5% 


90.3% 


91.1% 


93.1% 


AFDC case head less than 2 years 


6.9 


8.3 


6.6 


6,2 


AFDC case head 2 years or more 


1.7 


1.4 


2.3 


0,7 


JTPA required for welfare, food 










stamps, or WIN program ^ 


9.6% 


9.1% 


11.3% 


6.8% 


Household composition 










No spouse or own child present 


55.8% 


53.5% 


52.6% 


59.2% 


QyMn child under age 4, 










no spouse » present 


3.8 


5.2 


2.6 


2,8 


Own child, none under 4, 










no spouse, present 


4.6 


3.6 


3.8 


4.6 


Spouse present, with or 










without owTi child 


35.8 


37.8 


41.0 


33,4 


Family income In past 12 months 










< S3,000 


35.8% 


33.4% 


34.5% 


40,4% 


$3,000 - $6,000 


25.1 


26.0 


25.9 


22,5 


$6,001 - $9,000 


16.2 


16.8 


16.5 


14.6 


> $9,000 


23.1 


23,8 


23.2 


22.4 


Living in public housing 










Yes 


630.0% 


6.5% 


5.6% 


7.4% 


No 


93.7 


93.5 


94.4 


92,6 


Age at random assignment 










22-29 


44.6% 


48.1% 


44.0% 


42,4% 


30-44 


43.3 


41,5 


44.1 


43.6 


45-54 


7.8 


7.4 


8,0 


7.9 


> 54 


4.3 


3,0 


3,9 


6,1 


Mean 


33. J 


32.2 


33.1 


33.8 


Sample size 


3.759 


925 


1.832 


1.002 



Source: Unadjutied frcquencica baMd oo B*ckf round Infonmtion Foim fvtponftcs. 
a AFDC, GcaenU AMbunoe. of other welfare except food stampa. 

b. 'Any public auittaooe' includes the foUowinc touicca of aaniitance: AFDC. food suinpt. unemployment insuxanoe. 

houtinf aMiiiancc. md other caib aMiflancc. 
r. 'Other public aatiatanoe' includet unemployment tnaunnoc. houtin^ aaaiilance. and other (non*AFDC) cash 

aatiitance. 

d. WIN i« the fed^im) Woric Incentive prognun. 
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(versus 32.4 percent for the OJT/JSA and 34.6 percent for the other services 
subgroup). This subgroup was in-between the other uvo in terms of the 
proportion never previously employed (9.6 percent versus 6.1 percent and 12.6 
percent). Along with the OJT/JSA subgroup, members of the classroom trammg 
subgroup were more apt (at about 22 percent) to have earned a low wage (less 
than $4 hourly) in their most recent job than members of the other services 
subgroup, and more apt (at about 40 percent) to be receiving pubhc assistance 
than the other services subgroup. And as indicated in Exhibit ..17 in Chapter 
3 the classroom training subgroup was also the least employable or job-reads' 
of adult men, as measured by the estimated average earnings of the co"tr°l group 
over the 18-month follow-up period. Those estimates were $11,780 for the 
classroom training subgroup versus $12,456 and $12,516 for the other two 
subgroups, respectively. 

. The OJT/JSA service strategy was recommended for almost half of the 
adult male sample, and as a result, the OJT/JSA subgroup was by far the 
largest of the three (Exhibit 3.16 in Chapter 3). It also had the highest 
proportion of white men (68.9 percent) and the lowest proportion of 
black men ( 17.6 percent); the other two subgroups were about 40 percent 
black And as mentioned above, the OJT/JSA subgroup also had the 
lowest proportion of members who had never been previously employed 
(6.1 percent). i» 

. The other services subgroup had the lowest educational level, with 34 .6 percent 
without a high school credential (versus 25.2 percent and 32.4 percent for the 
othertwogroups). At 87.4 percentthis subgroup was also theleast likely to have 

been employed in the past; and its members earned the least, on average over the 
12 months preceding their application ($3,883 versus $4,000 and $4,01 1)^ The 
other services subgroup was also the most apt to have had a family income below 
$3 000 in the previous 12 months (40.4 percent versus 33.3 percent and ^AA 
percent) On the other hand, this subgroup was by far the least likely to be 
receiving public assistance upon application to JTPA (30.7 percent versus about 
40 percent for the other two groups). 

In addition to these observed differences it is possible, indeed likely, that the screening 
process used to recommend sample members for the different program services that defined 
each service strategy produced groups that differed in important ways that were not obsen'ed. 

Note however thai ll,e data in Bloom (1991) cover all JTPA appl.canis randomly assigned lo "caunem or 
™i sums w^^^^^^ Exhibu 5.9 includes only ihe aduU male treatment group m the smaller 1 8-month study 
sT;! A^indix A m the present report compares the treatment and control groups >n tins sample. 
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Hence, one could cannot distinguish between differences in program impacts among the 
three service strategies that were due to differences in their service mixes versus those that 
were due to differences in the t>pes of persons they served. 

As a result, one cannot use the estimated impacts for each service strategy to determine 
what would happen if a group of men were switched from one service strateg\' to another. As 
discussed in Chapter 2, to provide this information would have required a different 
experimental design that could only have been implemented within a special demonstration 
program and thus would not have reflected the experience of ongoing JTPA programs. The 
impact findings for each service strategy therefore indicate only the effectiveness of each for 
the group it was serving at the time of the study. 

Monthly Earnings Trends: Treatment and Control Groups 
WITHIN Each Service Strategy Subgroup 

Exhibit 5.10 displays the monthly earnings trends of treatment and control groups in each 
service strategy subgroup over the course of the foUow-up period. The earnings of all three 
control groups increased markedly over time; thus, wc can conclude that the earnings of the 
treatment groups would have increased without access to JTPA. This outcome, once again, 
underscores the importance of comparing treatment group outcomes to a valid estimate of 
what they would have been without access to the program. 

The largest and most consistent treatment-control group difference in earnings was in the 
OJT/JSA subgroup. This difference \\'as negligible for the first several months, but thereafter 
the treatment group earned consistently and substantially more, on average, than the control 
group. 

The pattern in the classroom training subgroup was quite different. There the treatment 
group earned less than the control group during the first several months, presumabh' while 
many treatment group members were in training and hence not available for flill-time 
emplojinent . For the next several months treatment group members earned more than control 
group members. But this advantage soon declined and fluctuated during the remainder of the 
follow-up period. 

In the other service subgroup, there was very little difference between the monthly 
earnings of treatment group members and control group members throughout the follow-up 
period. 
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Impacts on Earnings: JTPA Assignees and Enrollees in Each 
Service Strategy Subgroup 

Exhibit 5.11 presents quarterly and 18-month impacts on the earnings of assignees and 
enroUees in each service strateg}' subgroup.^^ The OJT/JSA strateg\' appears to have been 
the most successful for the men it served. 

There was a stable pattern of positive impacts on earnings for this subgroup, with no 
decline at the end of the period, as there was for the other two subgroups. The magnitude of 
the estimated impacts per JTPA assignee ranged from $54, or 3. 1 percent, in the first quarter 
to $201, or 9.3 percent, in the sixth. The estimates were statistically significant at 
conventional levels in the third and sixth quarters. 

The cumulative effect of these positive quarterly impacts for the OJT/JSA subgroup was 
an estimated impact on earnings of $78 1, or 6.3 percent, per JTPA assignee over all quarters. 
This result was statistically significant and implied a total impact per JTPA enrollee of 
$1,418. 

This evidence of a positive impact on earnings is reinforced by the significant program 
impacts on emplovTnent in the OJT/JSA subgroup to be presented in the next subsection. It 
therefore appears that adult men recommended for this service strategy experienced a real 
program-induced increase in earnings that became salient during the second follow-up 
quarter, was sustained throughout the remainder of the follow-up period, and produced a 
moderate overall cumulative increase in earnings for the follow-up period. 

In contrast, impacts on the earnings of men recommended for the other two service 
strategies were small, and inconsistent over time. The estimated total impact on earnings for 
all quarters was $4 1 8, or 3 .5 percent, for men in the classroom training subgroup and $26 1 , 
or 2.1 percent, for men in the other services subgroup. Neither estimate was statistically 
significant. Furthennore, no single estimate ofa quarterly impact on earnings was statistically 
significant for either subgroup, and in both groups there was no clear pattern to the estimates. 

Impacts on Employment: JTPA Assignees in Each Service 
Strategy Subgroup 

Exhibit 5.12 presents estimates of JTPA impacts on employment, as measured by the 
percentage employed, the average number of weeks worked and the average number of 

24. As noicd earlier, the earnings csiimaies include wages paid to JTPA participants in OJT positions 
During the 18-monlh follow-up period llic 16 SDAs reimbursed employers a total of about $700 per adult male 
OJT panicipant. Within the OJT/JSA subgroup the reimbursement was about S 1 70 per adult male assignee 
over the follow-up fxiriod. 
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Exhibit 5 a: 



Period 



Quarter 1 
2 
3 
4 
5 
6 

All quarters 



Quarter 1 

3 
4 
5 
6 

AU quarters 



Quarter 



1 
2 
3 
4 
5 
6 

All quarters 



hnpacis on Earnings: JTPA Assignees and Enrollees. by' S^rxice Strategy Subgroup 



Control 
mean 

(1) 



1.440 
1,714 
1,884 
2,184 
2,171 
2.387 
11,780 



1.757 
2,014 
2,133 
2,199 
2,183 
2.169 
12,456 



1,677 
1,951 
2,123 
2.199 
2 292 
2,274 
12.516 



Impact per assignee 



In$ 

(2) 



As % of (I) 
(S) 



Classroom training subgroup 

$ -101 -7.0% 

126 7.3 

213 11.3 

50 2.3 
151 6.9 

-21 -0.9 
418 3.5 



OJT/JSA subgroup 



54 
135 
164* 

94 
133 
201 
781* 



3.1% 

6.7 

7.7 

4.3 

6.1 

9.3 

6.3 



Other services subgroup 

74 4.4% 

104 5.3 

44 2.1 

44 2.0 

13 0.6 

-19 -0.8 

261 2.1 



Inferred impact per 
enrollee, in $ 

(4) 



-146 
183 
310 
73 
219 
-30 
608 



99 
244 
297 
171 
241 
366 
1.418 



131 
185 
79 
78 
23 
-33 
463 



Source: Estimates based on First Foliow-up Survey responses. - -lo*;. oit/tqa 

Notes- Sample size, classroom training subgroup: assignees = 732, control group - a'lo ^.^ntrnl orniin 
TuhpLD assim^ 1516 control group = 734; other services subgroup: assignees = 732. control group 
I I^O^BS^a^ rcg;cs^in-adjuslSl to\x>ntrol for differences in baseUne characlenslics between the 

^SS'sig^^^^^^^^^ i^eC^a?L^.05 level, at the .01 level (two-tailed test). 

IrgmS^^^^^^^ T column 3 are identical to those in column 2. Tests of statistical significance were 
not performed for olumn 4. 
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Exhibit 5.12 



Impacts on the Percentage Employed and on the Mean Number of ^'eeks 
and Hours Worked: JTPA Assignees, by Service Strategy Subgroup 





Percentage employed 


Weeks worked 


Hours worked 




Control 


Impact. 


Control 


Impact, 


Control 


Impac. 




mean 


in % pts. 


mean 


in weeks 


mean 


in hours 


Period 


(2) 


(2) 


0) 


(4) 


(5) 


(6) 








Classroom training subgroup 






Quarter 1 


50.5% 


-1.3% 


5.9 


-0.8** 


235 


-25 


2 


55.3 


3.8 


6.5 


0.2 


270 


16 


3 


60.7 


4.6 


7.1 


0.5 


293 


29 


4 


68.4 


1.2 


7.8 


0.3 


326 


12 


5 


72.0 


0.8 


7.8 


0.6 


315 


33* 


6 


73.7 


1.2 


8.8 


0.5 


341 


10 


All quarters ^ 


83.2 


1.3 


43.8 


1.3 


1,780 


74 








OJT/JSA subgroup 






Quarter 1 


56.9% 


4.5%** 


6.7 


0.4 


284 


12 


2 


61.3 


6.0*** 


7.6 


0.6** 


319 


21* 


3 


63.1 


6.5*** 


7.8 


0.6** 


329 


26** 


4 


66.8 


3.3* 


8.0 


0.2 


338 


10 


5 


69.3 


1.4 


8.0 


0.4 


334 


20* 


6 


68.3 


2.2 


8.7 


0.5 


325 




All quarters ^ 


83.8 


3.9** 


46.6 


2.2** 


1,929 


121* 








Other services subgroup 






Quarter 1 


65.9% 


-1.8% 


6.5 


-0.2 


264 


-7 


2 


65.9 


-0.2 


7.2 


0.1 


301 




3 


66.4 


3.6 


7.4 


0.3 


309 


5 


4 


66.0 


5.9** 


7.3 


0.6* 


311 


11 


5 


72.6 


3.1 


7.6 


0.5 


317 


15 


6 


66.9 


1.9 


7.7 


0.4 


313 


9 


All quarters ^ 


85.6 


1.4 


43.8 


1.7 


1.814 


31 



Source: Eaimates based oo First Follow-up Survey retponses. 

Notes: Sample size, clauroom traioiof subgfx>up: tstigDoes « 732, cootrol group * 325; OJT/JSA aibgroup: 
assignees * 1,516, control group « 734; other services subgroup: assignees « 732, cootrol group ■= 380. 
Estimates are regression-adjutted to cootrol for differeoces io baselioe cbarvcteristics betweeo the treatment 
group and cootrol group; see Appendix D. 

0. For columns I and 2 ('percentage employed') this row shows the percentage of cootrol group members who 
reported being employed at sjiy time during the follow-up period and the estimated impaa oo this percenuge 
for assignees, respectively. 

* Sutistically significant at the . 10 level, at the .05 level, at the .01 level (two-tailed test). 
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hours worked. All three sets of estimates indicate sizable and statistically significant impacts 
for the OJT/JSA service strategy subgroup and negligible impacts for the other two ser%'ice 
strategies, mirroring the earnings results in the preceding subsection. 

Specifically, estimates of program impacts on hours worked by men recommended for 
OJT/JSA (column 6) were statistically significant for all foUow-up quarters and did not 
diminish over time. Indeed, as was the ca-sc for impacts on earnings, the largest impact on 
hours worked by this group was in the sixth follow-up quarter. The cumulative effect of these 
impacts was a statistically significant total increase of 121 hours per assignee. Estimates of 
impacts on weeks worked by this group (column 4) were statistically significant in two of the 
six quarters and also were consistent over time, with a statistically significant 2.2 week total 
increase per assignee for the fiill follow-up period. Finally, estimated impacts on the 
percentage employed were statistically significant in the first four quarters but were 
substantially smaller and not significant in the last two quarters. For the foHow-up period 
as a whole the program increased the percentage of sample members employed b> a 
statistically significant 3.9 percentage points. 

In sharp contrast, the impact estimates for all three employment measures for the two other 
service strategy subgroups were small, and only 3 of the 42 estimates in columns 2, 4, and 
6 were statistically significant. 



Impacts on the Components of Earnings: JTPA Assignees 
IN Each Service Strategy Subgroup 

Exhibit 5.13 presents, for each service strategy subgroup, --stimates of JTPA impacts on the 
four components of earnings discussed earlier Because impacts on earnings (column 1 ) were 
statistically significant only for OJT/JSA (see Exhibit 5.11), the following discussion 
considers only this service strategy subgroup." 

The 6.3 percent increase in total earnings during the follow-up period for the OJT/JSA 
subgroup was due mostly to the 4.7 percent increase in workers per assignee.^* In other words, 
the earnings gain experienced by this treatment group reflected mainly an increase in the 
proportion who were able to find jobs. The other components of the group's earnings gain 
was a 1 .5 percent increase in the average nur.ber of hours worked per week worked. In other 

25. See Appendix D for a description of the method by which the estimates were calculated. Again, 
did not calculate significance levels for the earnings components exhibits because the estimates in the last three 
columns were calculated indirectly. 

26 Note that the employment impact in Exhibit 5. 1 3 is expressed as a percenUge of the corresponding 
control group mean In the preceding exhibit, 5.12 (column 2), the employment impact is expressed as a 
percentage point difference from the con-esponding control group mean. 
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Exhibit 5.13 Percentage Impacts on Earnings and lis Components: JTPA Assignees, 
by Service Strategy Subgroup 

Workers Wffks Hours worked Earnings 

Earnings per worked per week per hour 

per assignee assignee per worker worked worked 

Period (I) (2) (V (4) (5) 



Classroom training subgroup 



Quarter 1 


•7.0% 


-2.6% 


-10.5% 


2,4% 


4.3% 


2 


7.3 


6.8 


-3.2 


2.3 


1.4 


3 


11.3 


7.6 


0.0 


2.1 


1.2 


4 


2.3 


1.7 


1.7 


0.1 


-1.2 


5 


6.9 


1.1 


5.9 


3.2 


-3.2 


6 


-0.9 


1.6 


3.5 


-2.1 


-3.7 


All quarters 


3.5 


1.6 


1.3 


1.2 


-0.6 








OJT/JSA subgroup 






Quarter 1 


3.1% 


7.9% 


-1.7% 


-1.8% 


-1.0% 


O 


6.7 


9.8 


-2.1 


-0.8 


0.0 


3 


7.7 


10.3 


-2.3 


0.2 


-0.3 


4 


4.3 


5.0 


-2.1 


0.2 


1.3 


5 


6.1 


2.0 


2.9 


0.9 


0.1 


6 


9.3 


3.2 


2.6 


3.8 


-0.6 


All quarters 


6.3 


4.7 


-0.1 


1.5 


0.0 








Other services subgroup 






Quarter 1 


4.4% 


-2.7% 


-0.2% 


0.3% 


7.2% 


2 


5.3 


-0.4 


2.2 


-2.4 


6.0 


3 


2.1 


5.4 


-1.8 


-1.8 


0.5 


4 


2.0 


9.0 


-C.2 


-4.8 


-1.5 


5 


0.6 


4.2 


1.7 


-1.2 


-4.0 


6 


-0.8 


2.8 


2.8 


-2.7 


-3.5 


All quarters 


2.1 


1.6 


2.3 


-2.2 


0.4 



Source: Estimates based on Pint Follow-up Survey response! . 

Noic«: Sample iizc, claisroom training tubgroup: autgnees « 732. control group = 325; OJT/JSA tubgroup: 
assignees = 1,516, contfol group ■ 734; other services subgroup: assignee* » 732, control group » 380. Estimates 
are regreui on-adjusted to coolrol for differences in baidtne cbaracteh sties between the ireatmem group and 
control group; see Appendix D. Columns 2 through 5 display the ittptci as a percentage of the corresponding 
control mean (not shown). For column 2 this means the impact on the employment me is calculated as a 
percenuge of the mean rale for the control group. Tests of statistical significance were not performed for 
any of the columns in this exhibit. 
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words, treatment group members who found emploN-ment tended to work slightl> more hours 
per week. There w-as virtually no effect on the average earnings per hour worked. 

Hence, the earnings gain experienced by the OJT/JSA subgroup was due entireh- to an 
increase in the amount of time worked and not at all to an increase in the wages received for 
the time worked. As mentioned earlier, however, this t>pe of finding for the subgroup overall 
could mask important compositional shifts \«thin the group of men who worked, and it was 
not possible to control statistically for such potential compositional shifts (see Appendix G). 

Impacts on Ezr .iigs: Adult Men in Selected Key Subgroups 

This section presents estimates of JTPA impacts on the earnings of selected key subgroups 
of adult men defined to represent segments of the JTPA Title II-A population that have 
received attention in polic> discussions about the program. Other key subgroups were defined 
to represent groups with dilferent labor market experiences and those expected to have 
different needs for employment and training services.^' 

Specifically the analysis was designed to identify groups for which the JTPA Title II- 
A program was effective or ineffective at the 16 study sites. Distinguishing those groups for 
which the program was working can facilitate fiiture research efforts to study the factors that 
lead to program success, while identifying groups for which the program was not working 
can help target efforts to improve it. 

By itself, however, the analysis cannot determine why the program was effective for some 
groups and not for others. And'it cannot yield simple prescriptions about how to improve the 
program. The analysis can only measure the effects of the program, the way it was actually 
operated, on the people it actually served. 

Exhibit 5.14 presents the findings for key subgroups of adult men in the same way that 
Exhibit 4. 1 5 in Chapter 4 did for subgroups of adult women. Each panel in the exhibit defines 
a set of subgroups in terms of a particular dimension, for example, ethnicity, selected barriers 
to emplo\ment, and work histories. 

The first column in the exhibit presents the sample size for each subgroup, including both 
treatment group and control group members. The second column displays the mean earnings 
of control group members in each subgroup over the IS-month follow-up period. The third 
column presents the estimated impacts per JTPA assignee in eachsubgroup. Asterisks beside 

21 We selected the subgroups examined in this section based on their relevance to policy discussions, 
before we calculated the estimates. In other words, we did not select them on the basis of the size or signiticance 
of tlie program effects presented below . 
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the impact estimates denote that the estimates were statistically significantl>- differeni from 
zero, whereas asterisks in the final row in each panel indicate that the subgroup impact 
estimates in the panel were statistically significantly different firom one another. 

The fourth column displays the column 3 estimates adjusted for differences in the 
distribution of the subgroup across the 16 study sites. These adjusted estimates control 
statistically for the extent to which some subgroups were more heavily conceniraied m sites 
with more or less positive impacts than others. The estimates in column 5 adjust the column 
3 estimates for both differences in subgroup distributions across sites and differences in 
subgroup distributions across service strategies.^ 

To interpret the findings in the exhibit, one should proceed as follows. First, to assess 
the likely variation in impacts across subgroups, compare the impact estimates in column 3 
Estimates that are statistically significant are those that are most likely to represent true 
impacts for a subgroup, as opposed to chance results due to random sampling error. In other 
words, one should place confidence in these estimates. Moreover, those panels that contain 
subgroup impact estimates that are statistically significantly different from one another 
provide the strongest evidence that the true impacts for the subgroups in the panel were 
actually different. If the subgroup impact estimates within a panel are not statisticaM}- 
significantly different, the fact that their point estimates vary does not offer sufficient evidence 
that the true impacts for the subgroups were actually different from one another. 

On the other hand, the fact that subgroup impact estimates are not statistical!}' significant, 
or not statistically significantly different from one ariother, does not necessarily mean that the 
impacts or the impact differences do not reflect real impacts or real differences. Because of 
the small sample size of many subgroups, the possibility of random sampling error in the 
impact estimates for those subgroups is ver>' high. In these cases the available data are not 
sufficient to determine with precision the magnitude of subgroup impacts or their differences. 

Having examined the column 3 estimates for a set of subgroups in a panel, one should 
next read across the rows to columns 4 and 5. To the extent that the variation in impact 
estimates across subgroups changes as one moves from the column 3 estimates to the adjusted 



28. The eslimales in columns 4 and 5 adjust the distribution of each subgroup lo equal the disU-ibuiion 
of adull men overall across sites (column 4) or across sites and service strategics (column 5). For a full 
description of the methodology for deriving these eslimales, see Appendix D. 

29. All the subgroup impact estimates in Exhibit 5. 14 are based on ordinary least squares regressions on 
a pooled sample of all adull men, v^ilh the ircalmenl indicator interacted with the defming characteristic of the 
subgroup and (as appropriate) site and site and service su-atcgy. This approach allowed direct calculation of 
the F-lest for diflerences in impacts among subgroups in each panel of the exhibit. Subgroup impacts were also 
estimated on samples containing only die subgroup of interest; in general, these estimates dilTcrcd little from 
those based on the pooled regressions. 
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Exhibit 5.14 Impacts on Total IS^Month Earnings: JTPA Assigfiees. by SeUcted Key Subgroup 

— ^ Impact, in $, adjusied for 

sample distribution across: 



Key subgroup, 
defined by: 



Sample 
size « 



Control 
mean 

a) 



Impact, in $ 
(3) 



Sites 
(4) 



Sites and service 
strategies 
(5) 



Ethnicity 

White, non-Hispanic 
Black, non-Hispanic 
Hispanic 

F-icsi, difference among subgroups 

Barriers to employment (in italic) 
Receiving cash welfare 
No cash welfare 

F-lcsi, difference between subgroups 
No high school diploma or 
GED certificate 
High school diploma or 
GED certificate 

F-lest, difference between subgroups 
Worked less than 13 weeks 
in past 12 montlis 
Worked 13 weeks or more 
in pasi 12 momhs 
F-tea, difference between subgroups 

Number of barriers 
None of the above 
One of the above 
Two of the above 
All three of the above 
F-tesi, difference amon^^ subgroups 

Work histories 
Never employed 
Earned < $4 hourly in lasi job 
Earned $4 or > hourly in last job 

F-test, difference among subgroups 
Employed upon application 
Noi employed upon application 

F-tesi, difference between subgroups 



2,668 


$ 12,929 


$ 625 


$ 807* 


S 769 


1,155 


10,931 


957 


73 


183 


400 


13,555 






454 






n.s. 


n.s. 


n.s. 


611 


9,541 


-46 


52 


38 


3,788 


13,032 


624* 


616 


608 




n.s. 


n.s. 


n.s. 






398 


447 


531 






878** 


923** 


928** 




n.s. 


n.s. 


n.s. 


1,614 


10,478 


-210 


-110 


-91 


2,392 


14,320 


787* 


743 


740 






n.s. 


n.s. 


n.s. 


1,465 


15,142 


1,203** 


1,199** 


1J58* 


1,550 


12,184 


194 


296 


332 


617 


9,044 


30 


181 


500 


116 


8,595 


-146 


-356 


-1,248 






n.s. 


n.s. 


n.s. 


365 


8,813 


142 


217 


482 


728 


8,692 


1,372 


1,406 


1,475* 


3,326 


13,640 


470 


482 


454 






n.s. 


n.s. 


n.s. 


619 


13,618 


2,093** 


2,017** 


1,761* 


3,782 


12,341 


290 


315 


376 






« 


• 


n.s. 



(Cormnued) 
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Exhibit 5,14 Impacts on Total IS-Month Earnings: JTPA Assignees, by Selected Key Subgroup 
(continued) 



Itnpaa, in $, adjusted for 





Satnplc 


Control 






Sites and service 


Key suo group f 


siz.e^ 






Sites 




QcjiTico oy. 






(3) 


(4) 


(5) 

1-'/ 


Household composition 












No spouse or own child present 






> Hot 






Own child under age 4, 












no spouse, present 




1 J, 








Own child, none under 4, 












no spouse, present 


179 


11.092 


1,200 


1.082 


1.340 


Spouse present, with or 












without own child 


1,592 


14,927 


464 


564 


585 


Fat^d diffipr^nt*^ Anv%no ci.ihffmi.inc 


1,323 


7,550 


n.s. 


n.s. 


n.s. 


Family income in past 12 months 












$6,000 or less 


2,432 


10,542 


530 


566 


553 


More than $6,000 


1,736 


15,750 


30 


-11 


-76 


F-tcfi, difference between subgroups 






n.s. 


n.s. 


n.s. 


Living in public housing 












Yea 


283 


10,946 


-495 


-474 


-319 


No 


4,021 


12,595 


695* 


711** 


685* 


F-iest, difference between subgroups 






n.s. 


n.s. 


n.s. 


Age at random assignment 












22-29 


1,955 


13,361 


704 


654 


665 


30-44 


1,925 


12,213 


693 


715 


674 


> 44 


539 


10.489 


-537 


.287 


-297 


F-tesi, difference among subgroups 






n.s. 


n.s. 


n.s. 


Recommended for JSA only 












Yes 


335 


13,727 


-658 


-286 


n/a 


No 


4,084 


12.425 


640* 


641* 


n/a 


F-test, difference between subgroups 






n.s. 


n.s. 


n.s. ' 


Full sample 


4,419 


12,306 


550 


550 


550 



Source: Etiimaies baicd on First FoUow-up Survey raponscs. 

Notes: HAinutes &re regression-adjutied to control for dtfTereooes io iMieline chsractenttics between the trestment {roup snd eonirot 

group: tee Appendix D. Control means were not regretsioD-adjusied. Sample sizes for ouituaily exelustve subgroups within a paitcl do 

not necessarily mm to the sample size for the lartet group as a whole, because persons in omitted subgroups or with missing data on the 

variable used to define the subgroup are excluded. 

a. Treatment and control groups combined. 

h. AFDC, General AssisUnce. or other welfare except food «amps. 

c. WIN is the federal Wort Incentive prognm. 

• Sutisiically stgnincant at the .10 level. at the .05 level. at the .01 level (F-tett or two-uiled t-tcst); "n.s." means the F-ictt for the 
difference in impacts between or Mnong the subgroups io the category is not statistically significant. 
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estimates in these last m-o columns, site-specific effects or site- and sei%'ice strategN -specific 
effects explain part of the subgroup variation. In other words, the fact that one or more of 
the groups were concentrated in different sites or different ser^icc strategies is responsible 
forpartofthevariationinsubgroupimpacts. Ontheotherhand, to theextentthatthe variation 
in impact estimates remains unchanged as one moves from the column 3 estimates to the 
adjusted estimates, differences in the distribuUons of the subgroups across sites or sites and 
service strategies do not explain the subgroup variations. 

The following discussion of the findings in Exhibit 5.14 focuses primarily on those 
subgroupimpact estimates that werestatisucallysignificantlydifferentfromzero and on those 

that were statistically significantly different from one another, because we can place the most 
confidence in these estimates. We begin, however, by discussing the first subgroup estimates 
in the exhibit, even though they were neither statistically significant nor significantly different 
from one another, because this lack of significance is an important finding to note. This set 
of estimates is for the major ethnic groups. 



Ethnic Groups 



Perhaps the most xvidely discussed subgroups among the JTPA-eligible population are those 
defined in terms of their ethnic backgrounds. In particular, there has been considerable debate 
about the extent to which JTPA is serving white versus minority persons and about the extent 
to which the^jrogram is providing different t>pes of services to each of these groups (see, for 
example, U. S. General Accounting Office, 1991). 

As sho%vn in the first four rows of Exhibit 5. 14, the point estimates for ethnic subgroups 
differ but the results do not indicate a consistent or statistically significant relationship 
between ethnic background and program impacts. None of the impact estimates m column 
3 for whites blacks, or Hispanics was statistically significant, nor were the three estimates 
statistically significantly different firom one another. When the estimates for blacks and 
Hispanics are adjusted for their distribution across sites, the point estimates change, reflecting 
the relative concentration of the subgroups in some sites (especially Hispanics), but in neither 
column 4 nor column 5 were the three estimates statisticalh' significantly different from one 
another even though the adjusted estimate in column 4 for white men was significant. All 
told, these findings do not provide strong evidence of major differences in the impacts for men 
in the three main ethnic groups. 



30 Unlike the results for adult women in Chapter 4, Uie difference among the nnpact estnnates for men 
in the three ethnic groups was rot statistically significant at a near-conventional level (the . 1 5 level. lw(viailed 
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Subgroups Facing Selected Barriers to Employment 

For this analysis we defined the selected barriers to emplo\Tnent facing sample members as 
welfare receipt (measured by whether they were receiving AFDC, General Assistance, or 
other welfare except food clamps when they applied to JTPA); limited education (measured 
by whether they lacked a high school diploiiia or GED certificate upon application); and 
limited recent work experience (measured by whether they had worked less than 13 weeks 
during the 12 months preceding their ^plication). 

The first finding to note here is the extent to >^^ich each employment barrier was reflected 
in the 1 8 -month follow-up earnings of control group members and, hence, in what treatment 
group members would have earned without access to JTPA. As shown in column 2, control 
group members who faced an employment barrier, earned much less than their counterpans 
who did not face that barrier. Therefore, as was the case for adult women in Chapter 4, these 
barriers were indeed obstacles to employment. 

Second, in each case those issignecs who did not face the barrier in question experienced 
a statistically significant positive impact on earnings. These estimated impacts ranged from 
$624 to $878. In contrast, assignees who were receiving welfare upon application, assignees 
with limited education, and assignees with limited recent work experience did not, on average, 
experience a significant positive impact. This pattern did not change materially when the 
estimates were adjusted for the distribution of sample members across sites or across sites 
and service strategies. 

In no case, however, was the difference between the estimated impacts for these pairs 
of subgroups statistically significant. Thus, the evidence suggests that men who were not 
facing a major employment barrier, and hence were presumably more job-ready than those 
who were facing such a barrier, experienced larger program-induced earnings gains, but we 
cannot be sure of these differences. 

Because some men facing one of these barriers to employment were also facing one or 
both of the others, the three subgroups overlap. To distinguish among subgroups by the 
overall difficulty they faced in becoming employed, we also estimated impacts on subgroups 
facing none, one, two, or all three of the barriers, as displayed in the next four rows of the 
panel. 

Once again, two findings stand out. First is the extent to which the barriers, as defined, 
predict the difficulty of becoming employed. Specifically, as the number of barriers 
increases, average control group earnings decrease, from $15, 142 to $8,595. This consistent 
and striking pattern clearly illustrates the importance of these barriers in determining labor 
market success. 
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Second is the fact that the only one of these subgroups to experience a siatisticalh- 
significant positive impact on earnings was that which was facing none of the three 
emplosment barriers. The impact estimates for the other subgroups were small and in one 
case negative; and none was statistically significant. This pattern was similar when the 
findings were adjusted for the distribution of subgroups across sites or sites and sen/ice 
strategies.^' Nevertheless, the difference in impact estimates across subgroups w-as not 
statistically significant. Thus, although we are confident there was a positive program impact 
on those men facing none of the three employment barriers, we cannot say this effect was 
greater than that on men who were facing one or more of the barriers. 

Subgroups Employed or Not Employed upon Application to JTPA 

Other subgroup findings that showed some statistical significance were those for subgroups 
of men defined by their employment status upon application to JTPA. As shown in the last 
two rows of the next panel, the estimated impact for men who were employed when thes- 
applied was $2,093, whereas that for men who were not employed was $290. The first 
estimate was statistically significant, and the second was not; and the two estimates were 
statistically significantly different fi-om each other. Moreover, the pattern is similar when the 
fmdings are adjusted for the distribution of sample members across sites or sites and service 
strategies (although the two estimates were not significantly different from each other in 
column 5). Thus, here we are fiilly confident that the program was working well for those 
men who were employed upon application and less well for those who were not. 



Subgroups Living in Public Housing or Not 

The large positive impact estimates for men who were not living in public housing when they 
applied to JTPA (4,021 persons) was statistically significant and varies little when it is 
adjusted for the subgroup's distribution across sites or sites and service strategies. There is 
thus strong evidence that JTPA produced a real, positive impact on earnings for this group 
and that the impact estimate does not reflect a differential distribution of the group across 
sites or service strategies. 

In contrast, the impact estimate for the much smaller sample of men who were living in 
public housing when they applied to the program (283 persons) was negative and not 
statistically significant. This estimate and the adjusted estimates in columns 4 and 5 were 
not statistically significantly different from those for the other subgroup, however. 



3 1 . The shift in column 5 to a large negative impact for mer with all three barriers to employment rcnccu 
large amount of sampling error due to the very small sample upon uiiich the estimate is based. 
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Men Recommended for Job Search Assistance Only 

In previous sections of this chapter we presented estimates of program impacts on the 
earnings of men recommended for each of the major JTPA service strategics. Here we 
present the estimated impact on the earnings of men recommended for job search assistance 
only. This low-intensit>' approach was recommended for about a third of the men in the other 
services subgroup. As shown in the last panel of the exhibit, the estimated impact on the 
eamingsof these men was statistically insignificant but negative, in contrast with the estimate 
for all other men. The difference in estimated impacts between the two groups was statistical ly 
insignificant, however, and the subgroup recommended for job search assistance onl\' was 
too small to estimate the impact precisely. 



A Summary of Impacts, in the Context of Previous Research 

This final section summarizes the preceding findings on JTPA Title II-A impacts on the 
earnings, emplovment, and high school attainment of adult men. We first review the findings 
for adult men overall and compare them with results fi-om previous experimental studies of 
emplovment and training programs for men. These findings serve as a benchmark for then 
considering our more detailed findings for men in the three service strateg>' subgroups and 
other selected key subgroups. Exhibit 5.15 displays our principal findings. 



Adult Men Overall 

Estimated impacts on the earnings of those adult men offered access to JTPA Title II-A 
programs averaged $550, or 4.5 percent of control group earnings, over the 1 8-month follow- 
up period. The program increased the percentage of these JTPA assignees who were 
employed at some time during the follow-up period by 2.8 percentage points; it increased the 
average number of weeks worked during the period by 4.8 percent; and it increased the average 
number of hours worked during the period by 4.5 percent. 

The estimated impact on earnings was not statistically significant at conventional levels. 
But since the estimated impacts on all three employment outcomes were significant (.05 level), 
the estimated impact on earnings probably reflects a true impact, rather than a chance result 
due to random sampling error. 



32. Sec column 1. As explained in Chapter 2, ii was only in the other services subgroup iliat sample 
members could be recommended for job search assistance alone. 
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Exhibit 5. 15 Summary o/Eshn^ted JTPA Impacts on Earnings a,ui Employment 
over the Full Follow-up Period: AduU Male JTPA Assigrxees in the 
18-Momh Study Sample, by Service Strategy Subgroup 



Impact per 
assignee on: 


All 
subgroups 
(1) 


Classroom 
training 
subgroup 
(2) 


OJT/ 
JSA 

(3) 


Oiher 
services 
subgroup 


Earnings 
In$ 

As 56 of control mean 
Parce&Uge employed 


S 550 
4.5% 
2.8%** 


S 418 
3.5% 

1.3% 


$ 781* 
6.3% 
3.9%** 


S 261 
2.1% 
1.4% 


Weeks worked 
In weeks 

As % of conirol mean 


2 2** 
4.8% 


1.3 
3.0% 


2.2** 
4.7% 


1.7 
3.9% 


Hours worked 

In hours 

As % of control mean 


84** 

4.5% 


74 

4.2% 


121* 
6.3% 


31 
1.7% 



Source- Eaimiics based on Fim FoUow-up Survey reipoMC*. 

S"'Se Size. d»sroo»u«ning subgroup: ^signees- 732; OJT/JSA»b,^^^^^ 
l«i,n«i= 1516 control troup - 734; o<ha lervice* aibgroup: us>toctt ' 732. control group « iVJ. 
^'J^ « 4«^~n to c«nt«,l for diffcnces in b.«lioc cb«ctcri«ics between the tre-ment 

r.!Sr:^X^Tz.T^^^-^PV<^<^- •n-i-P.ctisn.c^uredinpercec.agepoint.. 

t « the .10 ...el. » .t the .05 levd. - .be .01 level f^^J^J^^' '"^ 

inm.cti on eJnings, wedcs woriced. uKl hour, v«,Aed, the «gmf.c«.ce levd of e«:h MinMt 
e^^-t^ of «nt,ol ««n- i. the -me « thit of the corresponding e«,m«e expre»ed .n dollars, 
in weeks, or in houti. 



The impact on earnings appears to be attributable to an increase in employinent among 
adult male assignees overall, and not to an increase in their average wage rates. Perhaps the 
simplest way to see this is to note that the impact on both average earnings and the average 
number of hours worked was 4.5 percent. Nevertheless, these findings for the target group 
overall could be masking important shifts xvithin the group in terms of who became employed 
and how their wage rates did or did not change. 

The impact findings are expressed as average treatment-control group differences, or 
impacts per JTPA assignee. They directly reflect the impact of randomly assigning sample 
members to the treatment group instead of the control group. But only 60.8 percent of adult 
male treatment group members actually became enrolled in JTPA, and 2.2 percent of the 
control group members also became enrolled. Furthermore, many control group members 
received services from other existing programs. 

As a result the impacts per JTPA assignee presented in this chapter do not reflect the 
impacts of receiving one or more emplov-ment and training services versus having received 
no such services Instead, they reflect the impact of receiving some employment and traming 
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service versus receiving less service. Thus, the impact of JTP A depends not only on the JTPA 
services provided to men in the treatment group. It also depends on the ser\dces they w ould 
have received from other providers had they not had access to JTPA, which we measure b\ 
service receipt in the control group. 

Direct evidence of the control group's receipt of non-JTPA services was a\'ailable only 
for classroom training in occupational skills and basic education. In the classroom training 
subgroup, treatment group members were more likely to receive classroom training in 
occupational skills than control group members (40.1 percent versus 24.2 percent). They 
were also more likely to receive basic education (10.0 percent versus 4.9 percent). 

We are also fairly confident that JTPA increased the receipt of on-the-job training in all 
three subgroups, since this service is seldom provided by non-JTPA programs. We were 
unable to measure the extent to which JTPA increased the receipt of job search assistance or 
miscellaneous services. 

While access to JTPA substantially increased the incidence of receipt of occupational 
skills training, basic education, and OJT, the treatment-control differential in the average 
number of hours of occupational skills training per assignee for men in the classroom training 
subgroup was only 95 hours. Similarly, men in the OJT/JSA treatment group received only 
114 more hours of OJT per assignee than their control group counterparts. These relatively 
smaU differentials reflect the fact that not all treatment group members received these services 
and, in the case of occupational skills training, that a nonnegligible proportion of control group 
members received services. 

These service differentials led to a substantial increase in educational attainment for men 
in the classroom training subgroup, who were those most likely to receive occupational skills 
training or basic education. Twenty-seven percent of the high school dropouts in the 
classroom training treatment group attained a training-related high school credential, whereas 
only 1 1 percent of the control group members did so. Corresponding effects were much 
smaller for the other two service strategies, as would be expected from the nature of the 
services recommended for and received by members of these subgroups. 

Exhibit 5.16 compares the estimates of impacts on earnings for adult men in the National 
JTPA Study with corresponding impact estimates for adult men from previous randomized 
experiments. The exhibit identifies each study, describes the adult men in its sample, indicates 
the tNpes of program services they received, and presents estimates of program impacts. The 
impacts are stated as quarterly averages of the program impacts on the earnings of the 
treatment group, expressed in dollars and as a percentage of the control group mean. Separate 
impact estimates are presented for each of three years after random assignment, wherever 
available. To facilitate comparisons across studies, we converted all the estimates to constant 
July 1989 dollars. 
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Exhibit 5.16 Esnmaied Impacts on the Average Quarterly Earnings of Adult Men: 

The JTPA 18-Month Impact Analysis and Previous Experimental Studies 





Treatmmi 


Program 
services 


Quarterly $ impact per treatmem 
group member (% impact in parmihesesj 


(Year evaUuuion 
ktgan) 


0) 


a) 


Year I 

(3) 


Year 2« Year 3 


Naiional JTPA Study 
(198TJ 


Low'inoome mea, 6% 
recdvinf AFDC; 
pirUcipatioa volunury 


ClaMioom tminini 
•irmlegy: occupaiioeal 

aducatioa 


$72 
(4%) 


(3ft) 






OXT/JSAatraiefy: oo- 
tbe-job tfrnininc. job 
ff^rch aMitlaiioe 


«112 
(6«) 


$167* 
(8ft) 






Other tcrvioe* tiratciy: 
job teai-ch aiaiatance. 


$66 
0%) 


$-3 
(Oft) 






■uaccUiMOua 










All three Mfatcgioi 


$86 
(4%) 


$103 
(5ft) 


Sua Diego I 


AFDC applioaou; 
pirUcipatioa numdtioiy 


Job tcardi MaiitaDoe, 

uf^Mud work cxpericDoc 


$52 
(4%) 




San Diogo Saiuralioa ^ 
Work loitialivc Modd 
(1985) 


nctpienu; panicipation 
maadatoiy 


Job Bcarcfa aaaittaiicc. 
uopaid woric expcrioKX. 
bMic aducation. oocu- 
p^ional akiUa uminiof 


$142» 
(18ft) 


$125 
(12ft) 


Texas Worker AdJuM- ^ 
meot DctaonAration 

(1984) 

S'J. UI Reemployrocol 
DcmotutnUioD 


DUpUced woricen; 
participation volunUry 

DUplaoed woriccn; 
Participa£ioa mindaLof>' 


Job Bcardi aaaiManoc, 
occupational tkUk 
tfmuung. oo-the-job training 
Job March aaaistanoc 


$197/ 
(8ft) 

$239 
(•') 




(1986J 




Job March aMittanoe. 
occupational akilla trmininc. 
fclocation aatiMance 


$48 

(••) 








Job aearch aMtttaooe 


$150 








early recmployiDait bonus 


(••) 





Natiooal Suppjrted^VVork 
DcmonitralioD 

(1976) 



Lowinoome cx^ofTenden. 
95% of whom were men; 
participation volunUry 

Low ■ u^come ex-addicti. 

hO% of whom were men; 
paiticipatioo voKiMary 



Souciurad. paid worfc 
experience for m> to 
12 monUu 

StniciufDd. paid woric 
eiq>enenoe for i^) to 
l2monthc 



(7ft) 

$56 
(4ft) 



(17ft) 



$42 U 
(32%) 



Sourct 



Not 



+ Gueit« and P«uly (1991). Ubk 3 .1. pp 86-87. md labk 4.5. p. 161 . 

+ +Bioom (1990). tabk 8.1. p. 163, 

(198* ubir^ni. 3. p. 219): «1 Koivcr. Loo,. «d TWo dWl- -We V. 1 ««1 VII. 1. pp. 131 ..1 157) 



on the conuo! group wm not avaiUble from the •ouroc cited. 
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The existing previous experimental studies of employment and training programs serving 
men are limited to three subpopulations: low-income men who were receiving or had apphed 
for AFDC; displaced workers, who had lost formerly well-paid, stable jobs permanent!) 
because of changing technolog>' or increased foreign competition; and low-income men who 
were ex-offenders or ex-addicts. 

As shown in the second panel of Exhibit 5.16, two experimental studies focused on the 
effect of emplo>'ment and training programs for men who had applied for AFDC-UP, the 
component of AFDC for two-adult households with unemployed parents. The Emplo\'ment 
Preparation Program/Experimental Work Experience Program (commonly known as San 
Diego I) and the Saturation Work Initiative Model (SWIM) were both conducted in San 
Diego. Both programs provided a sequence of job search assistance followed by unpaid work 
experience for participants who did not find jobs. SWIM also provided basic education and 
occupational skills training for participants who did not become employed. The more 
intensive program, SWIM, produced quarterly impacts on treatment group earnings that were 
noticeable in the two years after random assignment and were statistically significant in the 
first year (at $142). The impact for San Diego I, the less intensive program, was small ($52) 
and not statistically significant in the first year— the only year for which impact estimates are 
available. 

The next panel presents mformation on the two experimental studies of male displaced 
workers. The Texas Worker Adjustment Demonstration, conducted in Houston and Ei Paso, 
provided a sequence of job search assistance followed by classroom training in occupational 
skills or on-the-job training for participants who did not find employment. The New Jersey 



a. The c<U mites from the NtUonil JTPA Siudy (firil panel) in this colunm are for the fir»l two quarters only. 

b. Basic education* includes Adult Basic Education (ABE), high school or General Educational Development (GED) 
preparation, and English as a Second Language (ESL). 

c. Miscellaneous" includes assessment, job-readincs* training, customized training, vocational exploration, 
job shadowing, and tjyout employment, among other services. 

d. "San Diego V is the common abbreviation of the San Diego Employment Preparation Program/Expcrimenul 
Work Experience Program. Conducted at all aeven welfare oiTiceft in San Diego County, the program had a net cost 
Cm nominal dollars) per Ireatinent group member of $636. 

€. The San Diego SWIM demonstration was conducted at two of the welfare offices in the county, 

at a net cost On nominal dollars) per trestmcnl group -member of $919. 
/ The Texas demonstration was conducted at one site in Houston and two sites in El Paso. Comparable 

information on the net cost of the program was not available from the source cited. 
g. The New Jersey demonstration was conducted at 10 unemployment insurance offices in the sutc. 

Comparable information on the net coat of the program waa not available from the aource cited. 
A. Included demonstrations in seven sites for ex-offenders and four sites for ex-addicU, with 

a net cost per treatment group member (in 1976 dollars) of $7,437 for ex^ffenders and $6,185 for ex-addicU, 

or $8,843 and $7,776 including revenues generated by the demonstration. 
I. Impact estimates are not reported for year 1 after random assignment becnuse most treatment group members 

were in the program and thereby receiving heavily subsidized earnings in that year. 
• Sutisiically significant at the .10 level, at the .05 level, at the .01 level (two-tailed test). 
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Unemplovment Insurance ReemploNTnent Demonstration Project, conducted at 1 0 Ul offices 
in the state offered three different t>pes of programs: job search assistance only; job search 
assistance combined wth occupational skills training or relocation assistance; and job search 
assistance combined with a cash bonus for early reemployment. Estimated program impacts 
in the first vear were modest but not statistically significant for the Texas program (SI 97) 
and for all three components of the New Jersey program (ranging from $48 to $239). 

The last study shown in the exhibit, the National Supported Work Demonstration, 
examined the impacts of a lengthy and intensive program of subsidized work expenence for 
adult male ex-offenders and ex-addicts. Because the program lasted more than a year for 
many sample members, only the findings for the second and third years after random 
assignment represent post-program impacts. These impacts were small and not statisticalK- 
significant in the second year after random assignment ($105 and $56) and larger for both 
groups, although still not statistically significant for ex-offenders, in the third year ($268 to 
$421). 

The impacts on treatment group earnings for adult men in the National JTPA Study are 
reported on a quarterly basis in the exhibit to facilitate comparisons with the findings from 
the other studies. The year 1 estimates are based on findings for the first four follow-up 
quarters, but the year 2 estimates are based on findings for the fifth and sixth follow-up 
quarters'only. The estimated impacts for men in the fiiU sample (all three service strategy 
subgroups) were $86 per quarter during the first follow-up year and $ 1 03 per quarter during 
the first half of the second year. These impacts are toward the low end of the findings from 
the previous studies. 

One must exercise caution in making these comparisons, however, because adult men in 
the JTPA sample differed substantially from those in the previous studies. For example, only 
6 percent of the men in our 18-month study sample were receiving AFDC upon application 
to JTPA and the A ^DC recipients in the San Diego programs were probably less job reads 
than the typical adult male in JTPA Title II-A. At the other extreme, the displaced workers 
in the Texas and New Jersey demonstrations had, by definition, more extensive emplox-ment 
experience than the typical Title II-A participant. (This difference was acknowledged b>' the 
establishment of a separate JTPA title. Title III, to provide services for displaced workers.) 
Finally, although we do not know the proportions of ex-offenders and ex-addicts among the 
men in our sample, we must assume they were small by comparison to those in Supported 
Work. " Moreover, the men in Supported Work received much more intensive services than 
those generally provided by JTPA. In the next subsection we will extend this comparison with 
previous studies in examining the findings for the three service strategy subgroups in our 
sample. 

33 As shov^n in Exhibit 3.5 in Chapter 3, for example, only 8 percent of all lerminees in JTPA 
Title n-A rationally during program years 1987-1989 were ever airested. 
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Men Recommended for Each Service Strategy 

As summarized in the earlier Exhibit 5.15, among the three service strategy subgroups the 
estimated impacts for the OJT/JSA subgroup were by far the largest and the onl\' ones that 
were statistically significant. 

At $781, the estimated impact on total 18-month earnings for the OJT/JSA subgroup 
was much larger than the estimates for the other two subgroups and the only one that was 
statistically significant. The same is true of the estimated impacts on all three employment 
outcomes. And as shown in Exhibit 5. 16, the average ^warrer/y impacts on earnings for the 
OJT/JSA subgroup ranged fi-om $ 1 1 2 in year 1 to (a statistically significant) $ 1 67 in the first 
half of year 2— toward the high end of the results from the previous experimental studies. 

The earnings impact for the OJT/JSA subgroup as a whole was produced almost entire!) 
by an increase in the amount of time worked, with virtually no impact on the average amount 
paid per hour worked. The simplest way to sec this is to note that the impact on both average 
earnings and the average number of hours worked was 6.3 percent (Exhibit 5.15). Once again, 
however, it is important to note that this finding for the OJT/JSA subgroup as a whole ma>' 
reflect program-induced shifts in the composition of the group that was employed. Hence, 
one cannot interpret the finding to mean that the subgroup experienced no effects on wage 
rates. 

The O JT/J S A impacts reflect a modest difference in the services received by the treatment 
group and the control group: Although 27.1 percent of the Ueatment group received on- 
the-job training fi-om JTPA and only 0.5 percent of the control group did so, this produced 
a difference of only 1 14 hours of OJT receipt per sample member beUveen the two groups. 
Because JTPA is typically the only provider of on-the-job training, this finding is probably 
a fair estimate of the actual difference in receipt of OJT. 

In addition, 30.7 percent of the OJT/JSA treatment group received ;ob search assistance 
from JTPA. Because JTPA ist\pically not the only source of this service, many control group 
members probably received this service from non-JTPA sources. There was, however, no 
adequate way to measure receipt of this service b\' control group members, and so we were 
unable to determine the corresponding treatment-control group difference. 



Men in Selected Key Subgroups 

Few of the impact estimates for key subgroups of adult men were statistically significant and 
even fewer were statistically significantly different from each other, including those for 
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different ethnic groups. Hence, the degree of uncertamt>- about these estimates and the 
relativelv small magnitudes of most of them provided few mdicat.ons that JTPA .mpact 
vanedsvstemafcallyacrosssubgroups. Nevertheless, one important pattern did emerge from 
the analysis: The largest JTPA impact on earnings was among those adult men who ucre 
the most job-ready. 

This relationship between job-readiness and program impacts emerged from t>^'o of the 
subgroupanalvsesreportedearlierinExhibit5.14. The first compared the estimated impacts 
on mlnwho. upon application, were facing none or one or more of the three selected barner 
toemploNTnent:welfarereceipt,limitededucation,and limited recentworkexpenence.Thos^ 

men facing none of those baniers experienced a large, statistically significant impact on 
earnings of $1,203 over the 18-month follow-up period, whereas the estimates for tnose men 
facing one or more barriers were small and statistically msignificant. 

Second the estimated JTPA impact was extremely large and statistically significant 
($2 093 per assignee) for men who were employed when they applied to JTPA. whereas the 
mpact was small and not statistically significant for men who were not employed at 
application. Moreover, the difference between these two estimates was staUstically sigmfi- 
cant in the F-test. 

The program also appeared to have served men who were not living in pubhc housing 
better than it served men who were. But here the estimated impact on the earnings of the 
former ($695) was much smaller than in the preceding two examples, and the difference 
between the estimates for the two subgroups was not significant, in part because the sample 
of men who were living in public housing was small. 



34 Because the difference in impacts among these groups was not statistically significant i" the F-test. 
u e c^of^s^^ the sub^oups facing one or more barriers did not experience a posm ve impact. But becau«. 
"c^mlJfoX subgroup facing no such bamer was highly significant, wc can be confident the estimate 
accurately reflects a positive impact on this subgroup. 



Out-of:School Youths: JTPA Impacts at 18 Months 



THIS ch3pXCT presents estimates of JlPA Title n*A inq)acts on outK)f*school youths. 
These estimates provide the first reliable evidence of the effects of a wide range of 
Title U-A youth programs. They also represent an inq)ortantcontributionto the small number 
of experimental studies of en^loyment and training programs serving young, econcmiicaliy 
disadvantaged Americans — ^^o, because of their limited work experience, are a very 
different population from adults and therefore should be studied separately. 

The 4,793 out-of-school youths in the IS-month study sample were 16 to 21 years 
old at their random assignment and fomied 28 percent of the full sample.^ With an average 
age of 19, this youth sample was over SO percent yM% 32 percent black, and 15 percent 

HispaJiic. 

These proportions were similar for both female and male youths, but on a variety of 
other characteristics the sample differed along gender lines. Starting with previous work 
experience, 23 percent of the females had never held a job before, vsiiereas only 16 percent 
of the males had never worked before; and 48 percent of the females versus 35 percent 
of the males either had never woriced or had earned less than $4 hourly on their most recent 
job. The females were somewhat better educated than the males, with 51 percent of the 



1 . Among the 2,649 female youths in the 1 S^onth study sample^ 2,323» or 87.7 percent, responded 
to the First Follow*-up Survey; among the 2, 144 male youths in the sample, 1 ,773, or 82.7 percent, responded 
to the survey. The estimated impacts presented in this chapXex are based on these respondent subsamples. 
The information on baseline characteristics in this paragri}^ is based on Background Inforaution Form 
responses for all out-of*school youths in the 18-month study sample. 
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former holding a high school diploma or GED certificate versus only 41 percent of the 
latter. Female youths, however, were much more likely to be living in a household receiving 
public assistance than male youths, at 47 percent versus 30 percent.^ 

The largest differences between the two genders, however, were in the proportion who 
were single parents and in the proportion who had been arrested. On the one hand, female 
youths were ahnost nine times more likely to be a single parent then male youths (at 36 
percent versus 4 percent); on the other hand, male youths were four times more likely than 
female youths to have been arrested before their application to JTPA (at 23 percent versus 
6 percent). 

As explained in Chapter 2, we had originally planned to analyze findings separately 
for white and minority youths. However, the impact findings presented in this chapter vary 
more by gender than by ethnicity.^ We therefore structure this chapter to contrast the 
estimates for female youths and rnale youths. There is some precedent for this distinction; 
previous research on employment and training programs for disadvantaged out-of-school 
youths has found different program impacts for female and male youths/ 

Moreover, our findings for youths served by JTPA Title II-A stand in sharp contrast 
to the gcjicrally positive impacts for adults presented in chapters 4 and 5. In short, the 
estimated impacts for out-of-school youths overall are negative, reflecting a combination 
of small, statistically insignificant negative impacts for female youths and large, statistically 
significant negative impacts for male youths, on average.^ 

The remainder of this ch^ter investigates these differential impacts in more detail. 
Because our splitting of the youth sample by gender resulted in twice as many findings 
to present as those in each of the two preceding ch^ters, we moved to Appendix H exhibits 



2. See Exhibit 3.14 in Chapter 3 for more detail. 

3. As shown in Exhibit H.21 in Appendix H, although the difference in impacts by gender was not 
statistically significant, the large, negative estimated impact for male youths was significant, whereas the 
small, negative estimated impact for female youths was not. These fmdings mean wc have confidence 
that the estimate for male youths reflects the true impact, but wc cannot be ture whether the estimate 
for female youths was significantly different from that for male youths. The con-e^Mmding estimated 
impacts by ethnicity were also not sUtistically significantly different from each other, but they di^layed 
smaller differences in magnitudes and sutistical significance levels than did the estimated impacts by 
gender. 

4. See, for example, Maynard (1980) and Cave and Doolittle (1991). 

5. The female and male youth samples in this chapter are about half the size of the samples of adult 
women and adult men. The two youth samples therefore have less sUtistical power, that is, they are 
less likely to yield impact estimates that are sUtistically significant In our discussions of the findings 
here, wc are therefore careful to say whether they were statistically significant or not. 
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corresponding to those in chapters 4 and 5 that were not essential to the discussion here.^ 
Otherwise the discussion roughly parallels the sequence of topics presented in those earlier* 
chapters. 

More specifically, the first main section examines impacts on earnings for out-of-school 
youths overall and by gender and compares those findings >vith estimates for young adults, 
ages 22 to 29. The second section presents more detailed estimates of impacts on the earnings 
of female youths recommended for each of the three service strategies, while the third section 
does the same for male youths. The fourth section examines impacts on female and male 
youths in selected key subgroups of interest to policymakers and program planners. In 
the final section we summarize the chapter's findings and place them in the context of results 
fi-om previous experimental research. 

In^acts on Earnings: 
Out-of-School Youths Overall and by Gender 

To clarify the differences between the findings for female and male youths, this section 
presents results for youths both overall and by gender. 

The first subsection uses monthly data to e?q)lore earnings trends over time, while the 
second examines JTPA enrolbnent patterns over time to distinguish between the in-prpgram 
and post*program periods for the treatment group. Trends in earnings and enrollment are 
of interest for three reasons. First, as we saw in the two preceding chapters, people tend 
to seek employment and training services at unusually low points in their earnings profiles, 
and so we can expect to see increases in control group earnings over time. Second, we 
also saw that some types of services (such as classroom training) involve an initial 
investment of time during which employment opportunities may be forgone, followed by 
a post-program period during which a return on that investment may emeige. Finally, among 
youths in particular, employment may increase over time as they mature. 

The third subsection shifts to an examination of quarterly estimates, presenting the 
estimated impacts on earnings per JTPA assignee (treatment group member) and per JTPA 
enrollee (treatment group member who became enrolled in the program at some time during 
the 18-month follow-up period). The last subsection compares the findings for the two 
youth target groups with those for their counterparts among the young adults in the sample 
to help determine \^ether the differences between the findings for youth and adults reflects 
a continuous trend between age and program inq)acts or a sharp change between impacts 
for the adult and youth programs. 

6, Supplementary exhibits in Appendix H for feroftle youths are those numbered H.l to H.6; for 
male youths, H.7 to H.12; and for all youths, H.13 to H.21. 
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Monthly Earnings Trends: Treatment Group and Control Group 

The first graph in Exhibit 6.1 shows that for aU out-of-school youths the earnings trends 
of the treatment and control groups were very different fi-om those for adults shown in 
the two previous chapter. Here the curve for treatment group earnings was below the 
curve for control group earnings in most months; for adults tiie treatment group curve was 
above the control group curve in most months.'' 

The othertwo graphs in Exhibit 6. 1 indicate that for femaleyouths control group earnings 
exceeded treatment group earnings to a small d^rec in most montiis, but for male youths 
the difference was much more pronounced and sustained. Among the males, earnings in 
the ueatment group were persistently bdow earnings in the control group, and the gap grew 
during months 6 to 9, dechned somewhat through month 14, then grew again. 

Indeed, as will be seen later, the treatment-control group difference in average eaniings 
among female youths was not statistically significant at convcnticmal levels. Nevertheless, 
the picture for female youths resembles in some ways the picture for adult women presented 
in Chapter 4, starting with an in-program period of for^cme earnings, during which treatment 
group members earned less than control group members, followed by a post-program period 
during which the treatment group earned slightly more than the control group, at least initially. 
In the case of female youths, however, the return on their investment was not sustained 
in the way it was for adult wOTicn; treatment group earnings dropped below control group 
earnings toward the end of the follow-up period. 

The trends in the control group's earnings also illustrate an important point; Even 
without JTPA the youths provided access to the program would have e?qperienccd substantial 
growthinmonthlyeamings. The estimates underlying the first grsqph (for all youths) indicate 
that this growth was from $313 inthe first month to $523 in the eighteenth, or an increase 
of 67 percent. Among female youths this growth was from $223 in the first mcmth to $399 
in the eighteenth, or a 79 percent increase, while for male youths it was from $425 to $684, 
or a 61 percent increase • 



7. Throughout this chapter, earnings and impact estimates arc expressed in nominal dollars. The 
follow-up period varied across individuals, beginning as early as November 1987 and ending as late as 
December 1990. 

8. Wc used ordinary least squares regression procedures to increase the statistical prectsiaii of th^ 
estimates, as described in Appendix D. The earnings estimates in Exhibit 6.1 and subsequent exhibits 
include wages paid to JTPA participants in on-the-job training positions. During the 18-inonth follow- 
up period the program reimbursed employers a total of about $650 ^female youth OJT participant and 
about $600 per male youth OJT participant. Among all youths in the sample OJT reimburscncnts over 
the follow-up period totaled about S60 per female youth treatment group member and about $80 per male 
youth treatment group member. 
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A last point to note before leaving Exhibit 6. 1 is the large difiference betv--een the earnings 
levels of females and males in every month during the follow-up period. The earnings 
of the male youth control group actually began in the first month at a higher level ($425) 
than the earnings of the female youth control group reached in the eighteenth month (S399). 
The large gender difference prevailed throughout the follow-up period, with the earnings 
of the male control group sometimes at a level almost twice that of the female control group. 



Enrollment Patterns over Time: Treatment Group 

As noted earlier, it is important to distinguish between the in-program and post-program 
periods for treatment group members, whose earnings may have been affected by forgone 
employment opportunities during the time they were enrolled in the program. 

Recall that we are using the first month in which less than 15 percent of the treatment 
group was still enrolled in JTPA as the beginning of the post-program period. Amorig 
youth treatment group members overall this point was in month 8 (sec Exhibit H.13 in 
Appendix H)— just beyond the start of the post-pr gram period for adult men (month 7) 
and earlier than the starting point for adult women (month 10). For female youths the post- 
program period started in month 9 (Exhibit H. 1), and for male youths, in month 7 (Exhibit 
H.7).' 

As shown in Chapter 3 (Exhibit 3 .20), the median duration of female youths' cnrolhncnt 
was more (at 3.9 months) than that of male youths (at 3.1 months). The corresponding 
figures for adults were 3.6 months for women and 2.5 months for men. 

Impacts on Earnings: JTPA Assignees and Enrollees 

Exhibit 6.2 quantifies the information in the last two graphs of Exhibit 6.1, aggregated 
into quarters and totals for the fiill 18 months.'*^ As shown in the top panel, the estimated 
program impacts on the earnings of female youth assignees (treatment group members) 
were negative during the three predominantly in-program quarters, virtually zero for the 



9. Tlic enrollment rates rqwrted in this paragraph arc based on data from SDA records that somewhat 
overstate the number of persons still enrolled in the program at any given time because the daU are missing 
some termination dates. Thus, Exhibit H. 1 3 serves as an upper bound on the percenuge of the treatment 
group still in the program in any given month; and the extent to which the graph oversUtcs the actual 
enrollment rates is higher in the later months. 

10. A similar exhibit for all youths combined appears in Appendix H (Exhibit H. 14). as do estimates 
of the effect of JTPA on the distribution of earnings for all youths (H.15), female youths (H.2), and male 
youths (H.8). 
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Exhibit 6.2 Impaas on Earnings: JTPA Assignees and Enrollees, by Gender 
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-9.2 


-300 


All quarters 


10,736 


-854** 


-7.9 


-1,356 



Source: Estimates b«sed on First Follow-up Survey re^nses. 

Notes: Sample size, female youths: assignees «* ljSt6, oontrol group * 714; male youths: assignees « 1,196, 
control group >■ 552. Esiimsfcs are regression-adjusted to control for differences in basdine rharacteristics 
becween the treatment group and control group; see Appendix D. 

* Statistically significant at the .10 level, ** at the .05 level, *** at the .01 level (two-tailed test). 



Significance levels for column 3 are identical to those in column 2. Tetts of statistical significance were 
not performed for column 4. 

fourth quarter, positive for the fifth, and then negative again during the sixth.^^ For the 
full 1 8-month follow-up period the estimated impact was $-1 82, or -2.9 percent of the control 
group mean of $6,225. None ofthese findings for female youths was statistically significant. 

The bottom panel of the exhibit shows strikingly negative results for male youths, unlike 
the small and statistically insignificant findings for female youths. The estimated impacts 
on earnings were negative in all six quarters and statistically significant in three. Over 
the follow-up period as a whole the average program impact was a statistically significant 
$-854, or -7.9 percent of the control group mean. As discussed later in this chapter, however, 

11 . To increase the statistical precision ofthese estimates, we used ordinary least squares regressiotis. 
This reduced the standard errors of the impact estimates but did not appreciably afifect the point estimates, 
because the average values of the independent variables (mainly the baseline characteristics of the treatment 
and control groups) were virtually the same for the two groups. See Appendix D for a description of 
these procedures. 
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the negative impact estimate for male youths overall reflects mainly a large, negative impact 
estimate for the subgroup of male youths who reported having been arrested between age 
16 and their random assignment.^^ For those male youths who had not been arrested the 
estimated impact was small ($-262) and not statistically significant. Thus, JTPA had 
similarly negligible impacts on most male youths as it did on female youths, overall, and 
an extremely negative impact on the subgroup of male youths with a previous arrest. 

Column 1, for the control group, shows what the male youths who had access to JTPA 
would have earned without the program: $10,736 over the 18 months— ahnost three-quarters 
more than the average for female youths. The control group e3q>erienccd strong growth 
in earnings overtime, from $1,412 in the first quarter to $2,063 in the sixth, or a 46 percent 
dicrease. The maricedly negative and sustained impacts shown in column 2 imply that having 
access to JTPA indeed reduced male youths' earnings, on average, at least during these 
first 18 months of follow-up. 

As shown in Exhibit 3. 12 of Chapter 3, not all youths who were given access to JTPA 
actually became enrolled in the program; 65.5 percent of female youth assignees and 66.8 
percent of male youth assignees were ultimately enrolled in the program during the follow- 
up period. Thus, as we did for adult women and men in the previous chapters, we also 
calculated inferred estimates of the program impact per JTPA enrollee}^ As shown in 
column 4, the inferred impact per female youth cnroUec was $-294, and the inferred impact 
per male youth cnrollee was $-1,349. Again, because of the uncertainty about whether or 
not treatment group nonenrollees experienced some impact from their contact with JTPA 
(see Appendix F), we did not attempt to calculate the statistical significance of the inferred 
impacts per enrollee. 

12. The extreme finding for this subgroup—* highly significant $-3,038 earnings loss— based on 
the First Follow-up Survey data is not supported by the impact estimate obtained from an alternative data 
source, namely, earnings data from state unemployment insurance agencies (sec Appendix E). These data 
show a negligible impact of $34 on male youths with a previous arrest and a small ($-240) negative impact 
on male youths overall, in contrast to the large, negative survey-based estimates. The Ul-bascd estimates 
arc derived from only four quarters of followmp earnings and arc also based on a smaller, nonraadom 
sample of the 18-month study sample. But neither data source produces results that suggest positive 
program impacts for male youths with or without a previous anesL Impact findings from the two data 
sources for the other three target groups— adult women, aduU men, and female youths— and for male 
youths without a previous arrest arc quite similar, however (see Appendix E). In our forthcoming final 
report we will investigate further the discrepancy in estimates from these two daU sources. 

13. As explained in Appendix D, the adjustment factor used to derive impacts per cmollcc from 
impacts per assignee is l/(r - c), \^ilerc r is the enrollment rate (the proportion of treatment group members 
who were enrolled in JTPA) and c is the crossover rate (the proportion of control group members who 
were enrolled in JTPA). Since these two rates are fixed for a given group or subgroup, the ratio of impacts 
per enrollee to impacts per assignee is also fixed for that group or subgroup. Thus, for example, for the 
female youth target group the impact per enrollee is 1.62 (1/1.655 - .0371) times the impact per assignee 
for all outcomes in all time periods^ \^ilile for the male youth target group the impact per enrollee is 
1.59 ([.668 - .038]) times the impact per assignee. 
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Impacts on Earnings: Youths and Young Adults 

The statistically insignificant $-182 earnings loss for female youths and the significant 
$-845 earnings loss for male y ouths differ substantially from the significant $539 earnings 
gain estimated for adult women and the near-ssgnificant $550 earnings gain estimated for 
adult men. One way to investigate possible sources of the different findings for youths 
is to examine whether there is an age-related pattern to the impacts. 

For example, if the differences in in^pacts between adults and youths were determined 
by age-related characteristics per se, one vfoM expect to sec a pattern of consistently 
less positive impacts in looking fi-om older young adults on down in age to the yoimgest 
youths in the sample. On the other hand, if adults and youths were recruited or served 
differently by JTPA Title II-A (and thus either participants or the program sendees differed 
substantially), one would expect to see an abrupt shift in impacts, from positive effects 
for young adults to negative effects for youths — ^at precisely Ae demarcation that defines 
' \e two groups, namely, between ages 22 and 21. 

Exhibit 6.3 allows us to investigate these possibilities by presenting separate impact 
estimates for each two-year age cohort of young adults (ages 22 to 29) and youths (ages 
1 6 to 2 1 ). The findings appear to indicate more of an abrupt shift from positive to negative 
impacts at the demarcation between adults and youths than a pattern of gradually diminishing 
impacts over the entire age range. Tests of statistical significance indicated, however, that 
the deviation of the impacts for youths from the trend in impacts for young adults was 
not statistically significant.'^ Thus, the findings inExhibit 6.3 arcinconclusive. Nevertheless, 
they at least suggest the possibility of some fundamental differences in the types of adults 
and out-of-school youths recruited to Title II-A or in the types of services the program 
delivered to these groups, or in both program recruits and program services. 



Impacts on Earnings and Its (Domponents: Female Youths 
Recommended for Each Service Strategy 

The impact estimates presented above are averages for all female and male out*of-school 
youths in the 18-month study sample. They therefore reflect the effects of JTPA Title 
n-A on a variety of different individuals whom program staff recommended for a number 



14. As shown in Exhibit H.22 in Appendix H, the estimated deviations from the young adult impact 
trend were S-169 for female youths ages 20 to 21 and S-702 for male youths ages 20 to 21. Estimates 
of the deviations from the young adult trend for youths ages 16 to 17 and youths ages IS to 19 were 
negative at a much greater magnitude (S-1,9I2 and S-2,1 1 8, respectively) but still sUtistically insignificant 
for males, and positive (S933 and S211, respectively) though again sUtisticaliy insignificant for females. 
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Exhibit 6.3 Impacts on Total IS-Month Earnings: JTPA Assignees Ages 16-29, by 
Gender and Two-Year Age Group 





Sample 


Mean earnings 


Impaa per assignee 




size!* 


Assignees 


Controls 


JnS 


As % of (3) 


Age group 


(V 


w 


(3) 


w 










Females 






16 • 17 


360 


$ 3,748 


% 3.643 


$ 105 


2.9% 


18 - 19 


1,040 


6,153 


6,437 


-284 


-4.4 


20-21 


900 


&,7o3 


7,157 






22-23 


724 


8,168 


7,595 


573 


7.5 


24-25 


740 


7,896 


7,951 


-55 


-0.7 


26-27 


713 


7,998 


7,327 


671 


9.2 


28-29 


653 


7,939 


6,565 


1,374** 


20.9 








Males 






16-17 


299 


$ 9,977 


$ 10,772 


$ -795 


-7.4% 


18-19 


754 


9,407 


10,776 


-1,369** 


-12.7 


20-21 


695 


11,598 


12,017 


^19 


-3.5 


22-23 


584 


13,805 


13,176 


629 


4.8 


24-25 


502 


13,358 


13,211 


148 


1.1 


26-27 


438 


13,106 


12,360 


746 


6.0 


28-29 


431 


13,983 


13,499 


484 


3.6 



Sourca: EiUmatci bued on Fint FoUow-up Survey fetpooiOi. The ctUmat«s for womea ^gei 22-29 «t 
raodom autpifflent are alto bai«d on earmngt data from ttaie uaenp^oymeot ioiuranoe (U1) ^genci«a. 
Notea: EatimatM are rqgTeatton-adjuated to control for differeooaa in baaelioe charartmaUca betwacn the 
treatment froup and control froup; aee Appendix D. 
a. Treatment and control ^roupa combined. 

* SuUstically ttpuficant at the .10 levd, at the .05 levc2» at the .01 level (two-tatled teat). Significance 
levels for column 5 are identical to tfaoae in column 4. 



of different employment and training services . This section examines the subgroups of female 
youths reconunended for each of the three service strategies, presenting, in turn, findings 
on service receipt, baseUne characteristics, and finally labor market impacts for each 
subgroup. 

Before turning to these findings, however, note that the service strategies that SDA staff 
recommended for youths differed fi'om those they recommended for adults. Service strategy 
recommendations also varied by gender. 

As shown in Exhibit 3.16 in Chapter 3, youths were more apt than adults to be 
recommended for the other services strategy — and less apt to be recommended for the 
OJT/JSA strategy. In the case of females the other services strategy was recommended 
for 32.5 percent of the youths but only 21.0 percent of the adults; whereas the OJT/JSA 
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Strategy vfzs deemed ^propriate for only 23.2 percent of the youths but 35.0 percent of 
the adults. Female youths and adult women were recommended for the classroom training 
strategy, however, in the same proportion: roughly 44 percent. And as with the two adult 
target groups, the main differences between the service recommendations for female and 
male youths were thatthe females were more likely to be recommended for classroom training, 
whereas the males were more likely to be recommended for OJT/JSA. 



Differences in Employment and Training Services Received: Treatment 
AND Control Groups within Each Service Stratfx?^' Subgroup 

As explained in Chapter 2, we define the impact estimates in this report as representing 
the incremental effect of the difference between the mix of services received by the treatment 
group and the mix received by the control group. Exhibit 6.4 allows us to assess the size 
of this increment by comparing the percentages of treatment and control group members 
in each service strategy subgroup who received each specific program service (our measure 
of the likelihood of receiving the service) and the mean number of hours of each service 
received (the amou?}f of the service received). The exhibit also allows usto assess differences 
in the types of services received across subgroups. 

The Classroom Training Subgroup. Although all members of the classroom training 
subgroup were recommended for classroom training in occupational skills, only 48.3 percent 
of the female youth treatment group in this subgroup reported receiving that service, as 
shown in the first panel of the exhibit.'^ The principal reason why some female youths 
did not receive that reconmiended service was that 28.S percent of the treatment group in 
this subgroup were never enrolled in JTPA during the follow-up period (Exhibit 3.12 in 
Chapter 3). 

The estimates of service receipt nevertheJess show that JTPA did increase the likelihood 
of receiving the two key services in this service strategy. Specifically, treatment group 
members were more likely than control group members to receive classroom training in 
occupational skills (48.3 percent versus 31.0 percent) and to receive basic education (16.S 
percent versus 12.2 percent). 



15. The survey-based estimates of receipt of classroom training in occupational skills and basic 
education shown in Exhibit 6.4 differ from the estimates based on SDA data in Cbi^ter 3 (Exhibit 3.18), 
for two reasons. First, the survey did not attempt to distinguish between services provided by JTPA and 
non-JTPA providers, whereas the SDA data cover JTPA-funded senices only. Second, the survey-based 
estimates are subject to respondent em)r in recalling and classifying services. In general, the SDA data 
are more reliable measures of JTPA services received. The survey-based estimates, however, are the only 
available measures of service receipt that include non-JTPA services and that are therefore comparable 
for the treatment and control groups. 
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Exhibit 6.4 Receipt ofEmploynunt and Training Services: Female Youxh Treatmenx Group and 
Control Group, by Sendee Strategy Subgroup 

Mean hours of service 



Specific program 
service 


Treattnent 
group 

0) 


Control 
group 

a) 


Difference, 
in % pts. 
(3) 


Treatmenx 
group 
(4) 


Control 
groiq> 

(5) 


in hours 

(6) 








Classroom traimng subgroup 






r^laftctw^m t Timing in 

occupationti akill/ * 
B&tic education 


48.3% 
16.5 


31.0% 
12.2 


17.3% 
4.3 


378 
73 


191 
37 


187 
36 


On-the-job tmning* * 

/ 1" I 'D A «r%lt*\ 

\J I r A OiUy) 


2.6 


0.0 


2.6 


24' 


0' 


< 

24 


Work experience 
I oiuy^ 


5.7 


1.0 


4.7 


c 

33 


c 

1 


c 

32 


Job seaich astisunce 














(JTPA only) 














Miscellaneout 














(JTPA only) 


7.7 


















OJT/JSA subgroup 






CUssroont timining in 

occupationAl ikills 
Basic education^ * 


17.9% 
5.4 


17.6% 
7.9 


0.3% 
-2.5 


101 
20 


117 
22 


-16 
-2 


On-the-job training* * 
(JTPA only) 


29.9 


0.5 


29.4 


HI' 


C 

6 


105 ' 


Work experience* * 
(JTPA only) 


5.2 


1.6 


3.7 


20 ' 


3 ' 


c 

17 


Job aearch assistance 














(JTPA only) 


28.3 












Miscellaneous * * 














(JTPA only) 


7.1 













(ConxsKued) 
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Exhibit 6.4 Receipt of Employment and Training Services: Female Youth Treament Group and 
Control Group, by Service Strategy Subgroup (continued) 



Percentage receiving service 



Mean hours of urvice 
per sample member 



Specific program 
service 



Treamient 
group 

(I) 



Control 
group 
(2) 



Difference, 
in % pts. 
(3) 



Treatment 
group 
(4) 



Control 
group 

(5) 



Difference, 
in hours 
(6) 



Other services subgroup 



Classroom training in 

occupational skills 
Basic education ^ 
On-the-job training^ * 

(JTPA only) 
Work experience 

(JTPA only) 
Job search assistance*^ * 

(JTPA only) 
Miscellaneou/**" * 

(JTPA only) 



24.9% 
23.1 

3.9 

3.4 
12.2 
28.5 



23.9% 
19.1 

0.4 

2.0 



1,0% 
4.0 

3.5 

1.4 



133 
70 

22' 

19 ' 



132 
67 

c 

16 



1 
3 

21 

3 



Sources: 

+ Uiudjusud frequencies in this row are baaed on Fir« Survey dau on roccipt of the service from any provider. 

+ + Unadjusted frequencies in this row are baaed on cnroUmen and trackinf daU from the 16 SDAs, the best available dau 

on receipt of this aetvice. Althouf h the dau are for JTPA Tide H-A-fundad aetvieas only, this service is typically not 

funded by non-JTPA provkien. 
+ + + Unadjusted frequencies in this row are alao baaed on SDA eorollmeflt and tracking daU. Companble daU on receipt of 

this service from other providen were not available; nor were comparable dau on receipt by cootrol froup mem b er s. 
Nous: Sample size, classroom training subfioup: treatmeot group «• 7D4, cootiot group » 341; OIT/JSA subgroi^: treatment 
group » 381, control group » 164; other servicea subgroi^): treatment group « 501, control group « 209. Because of missing 
dsu, sample sizes for services calculated from different dau sources may vary. Tesu of statistical significance were not 
performed for this exhibit. 

a. Lastii^ longer than one week. 

b. Lasting longer than one m- co. *Basic education* includes Adult Basic Education (ABE), high school or General 
Educational Developmcm i GED) preparation, and Engliih as a Second Laqguage (ESL). 

c. Hours, assuming a ^11 -time job at 40 hours per wsek. 

4. 'Miscellaneous* includes assessment, job-readtness training, cu Aomized tnioi^g, vocational exploration, job shadowing, 
and tryout employment, among other services. 



•? ^. c 
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The treatment-control group differences in the amount of each service received were 
187 additional hours of occupational skills training and 36 additional hours of basic education 
per treatment group member.** These differentials reflect differences both in the likelihood 
of receiving the two services and in the amount of the service received/?er recipient. Finally, 
note that job search assistance was also acommonly received service in the classroom training 
subgroup, probably as an adjunct to classroom services. 

The OJT/JSA Subgroup, As noted in chapters 4 and 5, we were unable to measure 
receipt of on-the-job training from non-JTP A providers. Nonetheless, the ueatment-conuol 
group difference in receipt of OJT from JTPA is probably a reasonably reliable indicator 
of the overall difference, since few non-JTPA providers offer this service. As shown in 
the second panel of Exhibit 6.4, among female youths recommended forthe OJT/JSA strategy \ 
only 29.9 percent of the treatment group received OJT, the defining service forthe strategy. 
Again, the main reason for nonreceipt was a failure to enroll in JTPA: 42.5 percent of 
the treatment group in this subgroup v;ere not enrolled (Exhibit 3,12 in Chapter 3). 

Merely 0.5 percent of the control group received OJT, however. Consequently, JTPA 
provided an additional 105 hours of OJT receipt, averaged over all treatment group members. 
Exhibit 6.4 shows that 28.3 percent of the OJT/JSA treatment group received job search 
assistance, the second key service in the OJT/JSA strategy. But because we could not 
measure the control group's receipt of this service, we could not determine the corresponding 
treatment-control group difference. 

The Other Services Subgroup. We were also unable to measure receipt of miscel- 
laneous services by the control group, and so we could not estimate the treatment-control 
group difference in receipt of this categoiy of services, ^ch was the most common category 
received by female youth treatment group members in the other services subgroup. We 
do know, however, that nearly a quarter of the female youth treatment group received 
classroom training in occupational skills and nearly a quarter received basic education, but 
treatment-control group differences in receipt of these services were small. The exhibit 
also shows that job search assistance was not a prominent service received by female youths 
in this subgroup. 



16. The hours shoun in columns 4 through 6 arc averages over all sample members, including those 
who were not enrolled in the service. To calculate the average number of hours of service receipt for 
only those sample members who actually received the service, divide the number of hours of service receipt 
per treatment group member or control group member (column 4 or 5) by the corresponding proportion 
receiving the service (column 1 or 2, divided by 100). 
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Summary of Differences in Service Receipt The preceding comparisons of service 
receipt across service strategy subgroups and between the treatment and control groups xsithin 
each subgroup can be summarized as follows: 

• Many female youth treatment group members did not receive the service 
recommended for them, either because they were never enrolled in JTPA or 
because they received a service other than the primary one recommended for 
them. 

• Nevertheless, the three service strategy subgroups represent distinctly different 
clusters of services actually received and are therefore useful for analyzing the 
effects of alternative service strategies. 

• Within each service strategy subgroup most female youths ^o were enrolled 
in JTPA received one or both of two key services, which varied by service strat^: 
In the classroom training subgroup these were classroom training in occupational 
skills and basic education; in the O JT/JS A subgroup they were on-the-job training 
and job search assistance; and in the other services subgroup they were mis- 
cellaneous services and basic education. 

• The services actually received by each service strategy subgroup were consistent 
with the recommendations of intake staff to a considerable extent, but ihey 
differed in some important ways. Most important was the unding that only 
29.9 percent of female youth treatment group members recommended for OJT 
(the 0 JT/JS A subgroup) actually received that service. Because 28.3 percent 
of female youth treatment group members recommended for OJT received job 
search assistance, it is most appropriate to characterize the OJT/JSA strategy 
as one based on placement in employment, with or without subsidized training. 

• Treatment-control group differences in the average amount of service received 
were relatively modest, where those differences could be measured. The average 
amount of additional occupational skills training recdved by treatment group 
members in the classroom training subgroup was 1S7 hours. Similarly, the 
treatment-control group difference in receipt of OJT by female youths in the 
OJT/JSA subgroup was only 105 hours. These modest service differentials 
reflect the &ct that not all treatment group members received these services 
and, in the case of occupational skills training, that some control group members 
received the service. 
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Exhibit 6.5 Impacts on Attainment of a Training-Related High School Diploma 
or GED Certificate: Female YouA JTPA Assignees Overall and 
High School Dropout Subgroup, by Service Strategy Subgroup 



Percentage attaining a training-related 
high school credential 





Sample 
size' 

(1) 


Control 
Assignees group 
(2) (3) 


Difference, 
in % pts. 

W 






Classroom training subgroup 




Full sample 


1,002 


15.4% 7.6% 


7.8%*** 


High school dropouts 


467 


32.9 16.6 


16.4*** 






OJT/JSA subgroup 




Full sample 


516 


3.4 1.9 


1.5 


High school dropouts 


172 


9.8 6.0 


3.8 






Other services subgroup 




Full sample 


679 


19.0 13.1 


5.9** 


High school dropouts 


411 


31.7 21.0 


10.7** 






All subgroups 




Full sample 


2,197 


13.7 7.9 


5.8*** 


High school dropouts 


1,050 


28.6 16.6 


11.9*** 



Sources: Unidjusted frequcoda based on Bftckground Infomwaion Fonn retpoiwa tnd 
First Follow-up Survey responses. ^ 

CL Trctimcnt md control groups combined. ' . • • ^ 

h, "Attainment of a tiaining-x«Utod lugh school otjdential" is defined as the combination of having received 

some school or training service and having attained a hi^ school diploma or General Education 

Development Certificate at some time during the l^monih follow-up period. 
• Staiiftially significant at the .10 level, *♦ at the .05 Icvd, at the .01 Icvd (two-taxied test). 

Impacts on High School Atttinmcnt As part of the training services reviewed above, 
a number of feniale youths gained a high school diploma or GED certificate. Exhibit 6.5 
displays program impacts on the attainment of a training-related high school credential 
which we dcfme as the percentage of the sample who reported both participating in a school 
or training program and attaining a high school credential at some time during the 18-month 
follow-up period. The exhibit reports impacts for both the sample overall and high school 
dropouts only. 

As can be seen, JTPA led to statistically significant gains in high school attainment 
for female youth assignees overall and for the dropouts, in both the classroom training and 
the other services subgroups— the two subgroups with relatively high rates of receiving 
basic education. These results were reflected in a significant 5.8 percentage point increase 
for assignees overall and a significant 1 1.9 percentage point increase for dropouts overall. 
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Differences in Baseline Chakacteristics across Service 
Strategy Subgroups: JTPA Assignees 

As explained in Chapter 2, '•/ariatioDS in impacts across service st^at^?gy subgroup'=' reflect 
not only the difference in tbs service each subgroup received, but also differences in the 
individuals recommended for each strategy. 

Columns 2 through 4 of Exhibit 6.6 show v^hy it is not possible to use the estonated 
impacts by service strategy subgroup to determine the bert service strategy for the average 
female out-of-school youtb in JTPA: because members of each of the three service strategy 
subgroups were quite different firom those in the other subgroups. 

By abnost every measure the 0.nVJSA :^iubgroup appeared to be tlie most employable 
and the other services subgroup to be the lesist employable, with the classroom t^afning 
subgroup in-between. For exampJe^ the OJT/JSA subgroup was the most ^t vo have a 
high school credential upon application (64.5 percent), to have worked before (£7.4 percent), 
and to be over age 19 (46.3 percent). The other services subgroap, cm the other hand, 
was the most likely to have woriced less than 13 weeks in the past 12 mouths (29.8 percent), 
to be receiving public assistance (54.2 percent), and to be living in a family with an annual 
income of less than $3,000 (52.4 percent). These differences were also reflected in the 
mean earnings of each subgroup over the year preceding ^plication: $1,795 for the 
OJT/JSA subgroup, $1,286 for the classroom training subgroup, and only $996 for the 
other services subgroup.'' 



Impacts on Earnings: JTPA Assignees and Enrollees in 
Each Service Strategy Subgroup 

As we have seen, both the characteristics of sample members and the services they received 
dififered among the three service strategy subgroups. Moreover, chapters 4 and 5 found 
different patterns of impacts for sample members deemed appropriate for the different service 
strategies. Separating the sample in this way may therefore provide some insight into the 



17. Exhibit 3.17 in Chapter 3, which displays mean earnings of the control gro^ over the 18*inonth 
follow-up period, shows similar differences among the three subgroups. For a more detailed deKtiption 
of the baseline characteristics of these subgroups, see Bloom (1991). Note, however, that the data in 
Bloom (1991) cover all JTPA applicants randomly assigned to treatment or control status, whereas Exhibit 
6.6 includes only the female youth treatment group in the smaller IS-month study sample. Appendix A 
in the present report compares the treatment and control groups in this sample. 
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Exhibit 6.6 Seleaed Baseline Characteristics: Female Youth JTPA Assignees, by Service 
Strategy Subgroup 



All 
subgroups 



Classroom 
training 
subgroup 



Characteristic 



om 

JSA 
subgroup 
(3) 



Other 
services 
subgroup 

(4) 



EUmidCy 

White, non-Hispanic 
Black, non-Hispanic 
Hispanic 

American Indian or Alaskan Native 
Asian or Pacific Islander 
Barriers to cmploymcot 

Q 

Receiving cash welfare 
No high school diploma or 

GED certificate 
Woiked leu than 13 weeks 

in past 12 months 
Number of barriers 

None of the above 

One of the above 

Two of the above 

All three of the above 
Work and training histories 
Ever employed 
Mean individual earnings 

in past 12 months 
Hourly earnings in last job 

Never employed 

Less than $4 

$4 or more 
Employed upon application 
Previously received occupational 

training 

Public assistance status ^ 

Receiving any public assistance 
Receiving AFDC 

Receiving food sUmps ^ 
Receiving other public assistance 



50.1% 


51.1% 


59.9% 


41.7% 


32.5 


24.8 


25.2 


48.1 


15.8 


22.4 


13.1 


8.8 


0.8 


0.5 


1.4 


0.8 


0.8 


1.2 


0.5 


0.5 


30.3* 






29.8% 


49.2 


48.2 




tin A 


60.5 


60.4 


46.4 


70.6 


20.8 


20.2 


33.6 


12.6 


32.3 


32.2 


35.9 


30.0 


32.3 


32.5 


22.7 


38.8 


14.6 


15.0 


7.8 


18.6 


76.9% 


75.0% 


87.4% 


72.0% 


$1,305 


$1,286 


$1,795 


S996 


23.1% 


25.0% 


12.6% 


28.0% 


47.6 


Ann 


51.2 


44.8 


29.3 


27.3 


36.2 


27.2 


14.4 


16.2 


16.9 


10.3 



25.5 

47.1% 
26.6 
39.3 
10.5 



21.5 

48.0% 
30.7 
38.9 
10.9 



33.4 

40.5% 
18.1 
31.6 
10.0 



25.3 

50.7% 
27.0 
45.6 
10.4 
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Exhibit 6,6 Selected Baseline Characteristics: Female Youth JTPA Assignees, by Service 
Strategy Subgroup (continued) 



Charaaeristic 



All 
subgroups 
(2) 



Classroom 
training 
subgroup 
(2) 



OJT/ 
JSA 
subgroup 
(3) 



Other 
services 
subgroup 
(4) 



AFDC history 

Never AFDC case head 

AFDC case head less than 2 years 

AFDC case head 2 years or more 

JTPA required for welfare, food 
sUmps, or WIN program ^ 

Household composition 

No spouse or own child present 

Own child any age, 
no spouse present 

Spouse present, with or 
without own child 
Family income in past 12 months 

< $3,000 

$3,000 . $6,000 

$6,001 - $9,000 

> $9,000 

Living in public housing 
Yes 
No 

Age at random assignment 
16- 19 
20-21 
Mean 

Arrested since age 16 ^ 

Yes 
No 

Sample size 



71.0% 
2L9 
7.2 

7.3% 

52.5% 

36.2 

11.4 

45.8% 
28.5 
9.8 
15.9 

13.9% 
86.1 

59.4% 

40.6 

19.0 

6.1% 
93.9 
1.814 



67.8% 
23.7 
8.5 

6.6% 

52.0% 

33.5 

14.5 

43.9% 
30.6 
9.9 
15.6 

12.6% 
87.4 

57.9% 

42.1 

19.0 

5.8% 
94.2 
803 



75.7% 
19.1 
5.2 

3.9% 

52.0% 

35.9 

12.1 

40.5% 
27.3 
10.6 
21.6 

11.4% 
88.6 

53.7% 

46.3 

19.4 

8.5% 
91.5 
421 



71.9% 
21.3 
6.S 

10.7% 

57.7% 

34.2 

8.1 

52.2% 
26.5 
9.2 
12.2 

17.3% 
82.7 

65.4% 

34.6 

18.8 

4.7% 
95.3 
590 



Source: Uiudjuitod frequencies bated oo Background Informition Fonn rejpomea. 
a. AFDC, Geoenl A«iMKe, or other welfare except food aumps. 

h. *Any public aaaiiuoce* iocludea the foUowinf aourcea of aaaiMDCc: AFDC, food aunpi, UMtrploymtflt inauraiice, 

houiinf aaaiAaoce, and other caih aHiitance. 
e, 'Other public aaaittaoce* includei unemployment inKirance, houaipf aaaiitance, and other (dod-AFDQ carii aaaitance. 
d WIN ii the fcdenl Work Incentive program. 

e. The percentage aneatad between age 16 and nndom aatignmem it baaed on reapooaea to the 

Pint FoIIow-up Survey by treatment group mcmbert who were included in the analysis of impacts on canui^s. 
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impact estimates for female youths overall. We do so in Exhibit 6.7.^* Although only 
a few of the estimates shown were statistically significant, the exhibit suggests some variation 
in impacts across the service strategy subgroups. 

More specifically, the classroom training strategy yielded large, statistically significant 
earnings losses during the first three, predominately in-program quarters, followed by small 
and statistically insignificant gains for the next two quarters, and then a small and insignificant 
loss in the last quarter. Over the period as a whole the classroom training subgroup had 
an earnings loss of $-542, or -9.1 percent of the control group mean of $5,936, although 
this estimate was not statistically significant. 

The OJT/JSA strategy, on the other hand, should not delay employment, and the exhibit 
suggests that no such delay occurred for the subgroup of female youths recommended for 
this strategy. Impacts were positive for the first five quarters but statistically significant 
only in quarter 2. Over the 18 months of follow-up the OJT/JSA subgroup gained $410, 
or 5.4 percent of the control mean of $7,620, but again this estimate was not statistically 
significant.'^ 

The other services strategy yielded an estimated impact of $-158, or 2.8 percent of 
the control mean of $5,726, over the 18 month follow-up period. The estimated impacts 
were small and negative in four of the six quarters but none was statistically significant. 

Impacts on the Componekts of Earnings: JTPA Assignees in 
Each Service Strategy Subgroup 

Exhibit 6.8 presents the findings of our decomposition of the percentage impact on earnings 
f r female youths into its four components: workers per assignee, weeks worked per worker, 
ours worked per week worked, and earnings per hour worked (columns 2 through 5). 
As explained in Chapter 4, each of these components reflects a different aspect of labor 
market success: the ability to find a job; how quickly assignees found jobs and how long 
they held them; the extent of full>time woric; and how much workers were paid for the time 
they woriced. Again, the estimates in columns 3 through 5 of the exhibit Zf>ply to only 
those sample members who were employed during the period in question* And the four 



18. Graphs of the monthly eunings trends on vvhich Exhibit 6.7 is based appear as Exhibit H.5 
in Appendix H. As noted earlier, the earnings and impact estimates include iMges paid to JTPA participants 
in OJT positions. The program reimbursed employers about $650 per female youUi OJT participant during 
the 18-month follow-up period. Within the OJT/JSA subgroup the reimbursement was about S160 per 
female youth assignee over the follow-up period. 

19. As noted earlier, the earnings estimates include wages paid to sample members placed in OJT 
positions. 



ERIC 



JTPA 18-MONTH IMPACTS / OUT-OF-SCHOOL YOUTHS ♦ 201 



Exhibit 6, 7 Impacts on Earnings: Female Youth JT^A Assignees and 
EnroUees, by Service Strategy Subgroup 





Control 


Impact 


per assignee 


Inferred impact per 




mean 


In$ 


As%of(l) 


enrdlee, in $ 




(1) 


(2) 


(S) 


(4) 






Classroom training subgroup 




Quarter 1 


$ 742 


$ -210*** 


-28.3% 


$ -307 


2 


909 


-189*** 


-20.8 


-276 


3 




-150* 


-14.2 




4 


991 


24 


2.4 


35 


5 


1,047 


70 


6.7 


102 


6 


1,196 


-87 


-7.3 


-127 


All iififlT^^rc 




-542 


-9.1 


-792 






OJT/JSA subgroup 




Quarter 1 


$ 1,002 


S 149 


14.9% 


$ 277 


2 


1,074 


203* 


18.9 


377 


3 


1,252 


97 


7.7 


180 


4 


1,363 


3 


0.2 


6 


5 


1,368 


103 


7.6 


191 


6 


1,562 


-146 


-9.3 


-271 


All quarters 


7,620 


410 


5.4 


762 






Other services subgroup 




Quarter 1 


$ 653 


S 43 


6.5% 


$ 74 


2 


909 


-68 


-7.4 


-117 


3 


1,023 


-96 


-9.4 


-165 


4 


1,047 


-52 


-5.0 


-89 


5 


1,093 


-41 


-3.8 


-70 


6 


1,001 


55 


5.6 


94 


All quarters 


5,726 


-158 


-2.8 


-271 



Source: EiUmatet bued oo FirH FoUow-up Survey mponaes. 

Notes: Sui^le nze, claaeroomtrainii^ fubgroup: tMigneee » 704, cocxrol froup •* 341; GJT/JSA tubffoup: 
•Migneet ^ 381, cootrol froup » 164; ocbtr tetvicee iubftoup: MUtgocu «« 501, cootiol group « 209. 
EAimitM aie regrcMioo-adjuked to cortrol for diflemicM to tMieline chAracteriftici between the 
tmtment group md conuo) group; tee Appendix D. 

* Sutifticelly agnficafU at the .10 level, at the .05 level, at the .01 level (iwtMailed teat). 
Significance levels for colunn 3 are ideolical to thoae in column 2. Teaii of MatiAscal aignficance were 
not performed for colunn 4. 
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Exhibit 6.8 Percentage Jmpaas on Earnings and Its Components: Female Youth 
JTPA Assignees, by Service Strategy Subgroup 



Workers Weeks Hours worked Earnings 

Earnings per worked per week per hour 

per assignee assignee per worker worked worked 

Fenod (I) (2) (S) W ffl 



Classroom training subgroup 

Quarter 1 -28.3% -19.7% -7.8% -2.8% -0.4% 

2 -20.8 -13.1 -7.9 -2.6 1.7 

3 -14.2 2.8 -10.8 -5.7 -0.8 

4 2.4 8.1 -3.5 -3.9 2.0 

5 6,7 1.7 1.6 0.3 3.0 

6 -7.3 1.2 -3.0 -1.9 -3.8 
All quarters -9.1 4.7 -11^5 -2^6 0 6 



OJT/JSA subgroup 

Quarter 1 14.9% 11.2% -5.1% 1.4% 7.2% 

2 18.9 10.3 -4.7 3.6 9.1 

3 7.7 2.2 -2.7 2.8 5.2 

4 0.2 -L9 -4.5 4.0 2.9 

5 7.6 -5.2 1.3 3.4 8.3 

6 -9.3 -9.0 -7.7 0.8 7.0 
All quarter* 5.4 4.3 -TJ 21 6.2 



Other services subgroup 

Quarter 1 6.5% 19.3% -8.9% -1.1% -1.0% 

2 -7.4 -4.8 -5.3 4.4 -1.8 

3 -9.4 -6.9 0.3 -2.2 -0.9 

4 -5.0 -1.8 2.9 1.8 -7.7 

5 -3.8 1.2 -1.5 0.8 -4.2 

6 5.6 1.0 0.3 2.2 1.7 
All quarters -2.8 3.4 -4^5 09 -2.4 



Source: Ettimttes baled on T'.rg, Follow-up Survey cttpofises. 



Notes: Sample size, cUssroom tnimnt wbtnup: assgnees ^ 704, cootrot group « 341; OIT/JSA subgroup: 
auigoees « 3S1, control group » 164; other tervtcet subgroup: asagnees « 501, oontiol group « 209. EaUnial et 
Mit regrettio&-adju«ed to control for differeaces in baseline characteristics between the treatment group and 
control group; tee Appendix D. Columns 2 through 5 diipUy the impact as a perceaUge of the corresponding 
control mean (not shown). For column 2 this means the impact on the employment rate is calculated as a 
percentage of the mean rate for the control group. Tests of Mistical stgairicance were not performed for any of 
the columns in this exhibit. 
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components are related to the estimated impact on earnings in column 1 by a simple 
multiplicative identity, which means that in each row of the exhibit the estimates in columns 
2 through 5 sum to roughly equal the estimate in column 

As shown in the top panel, in the classroom training subgroup the percentage impact 
on the employment rate (column 2) was clearly negative during the first two (in-prcgram) 
quarters, then appeared to turn positive or near zero in succeeding quarters. The estimated 
impacts on weeks worked per worker and hours worked per week worked were negative 
in every quarter but one. For all quarters combined the strongest negative factor in the 
-9.1 percent earnings loss was the -11.5 percent reduction in weeks woriced per worker. 
Thus, the negative impact on earnings in this subgroup appears to reflect primarily a reduction 
in how long sample members held jobs, rather than whether they found a job at all, how 
many hours they worked per week, or how much they were paid for the jobs they held. 

In the OJT/JSA subgroup the pattern was somcwiiat different. During the first two 
quarters more treatment group than control group members worked, as JTPA placed some 
treatment group members in OJT positions or regular jobs. That soon ended, however, 
and employment rates in the control group exceeded those in the treatment group during 
the last three quarters. Impacts on weeks worked per woricer and hours woriced per week 
worked present a somewhat contradictory story. Assignees who worked tended to work 
fewer weeks but more hours per week than control group members. The impacts on earnings 
per hour worked do show a pattern, however: consistently higher earnings among treatment 
group members who woilced than among control group members who worked. In sum, 
the initial boost in employmeni from the OJT/JSA strategy was gradually reversed, but 
treatment group members who worked ended up in better paying jobs on average but for 
shorter periods of time, yielding a S.4 percent gain in earnings per assignee. 

It is important to remember that, as explained in chapters 4 and S, the estimated impacts 
on earnings per hour worked may reflect program effects on the composition of the subgroup 
of female youths who were employed, in addition to, or instead of, any impact on the hourly 
earnings of specific individuals. In other words, the estimates in column S do not necessarily 
imply that JTPA increased or decreased the wage rates of individual workers.^^ 



20. For corresponding estimated impacts on female youths overall, see Exhibit H.3 in Appendix 
H. Exhibit H.3 in Appendix H presents impacts on the percentage employed and the number of weeks 
and hours worked for all female youths, including those v^o were never employed during the follow* 
up period. 

21. See Appendix G for a nonexperimental analysis, vvhich showed no statistically significant effects 
on the latent wage rates of female youths. 
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In the other service subgroupiht patterns were more complex, and noclcar story emerges, 
as might be expected from a strategy with such a broad mix of services. For the 18-month 
follow-up period overall the negative impacts on weeks woriced and hourly earnings per 
worker more than offset the positive impact on the employment rate, yielding a -2.8 percent 
loss in earnings per assignee. 



Impacts on Earnings and Its Components: Male Youths 
Recommended for Each Service Strategy 

This section examines the service receipt, baseline characteristics, and program impact 
findings for male youths in each of the three service strategy subgroups. The section will 
demonstrate that although the mix and amount of services received by male youths in each 
service strategy were similar to those received by their female youth counterparts, the males 
differed from the females in baseline characteristics, especially il their arrest rates and rates 
of single parenting. Moreover, the estimated program impacts by service strata subgroup 
also differed markedly by gender. 



Differences in Employment and Training Services Received: Treatment 
AND Control Groups within Each Service Strati oy Subgroup 

Before turning to the findings on service receipt by male youths, recall that youths were 
more likely than adults to be reconunendcd for the other services strategy and less likely 
to be recommended for the OJT/JSA strategy. And as with the adult target groups, the 
male youths were more txpi than the females to be recommended for OJT/JSA and less 
apt to be recommended for the classroom training strategy (see Exhibit 3.16, Chapter 3). 

The Classroom Training Subgroup. As shown in Exhibit 6.9, 42.6 percent of male 
youth treatment group members in the classroOTi training subgroup reported receiving 
classroom training in occupational skills, whereas only 22.4 percent of their control group 
counterparts did. Again, the principal reason why some Ueatment group members did not 
receive this defining service for the strategy was that 25.2 percent of the treatment group 
in this subgroup were never enrolled in JTPA (Exhibit 3. 12 in Chapter 3). Here the treatment- 
control group differential in the likelihood of receiving this service was similar to that for 
female youths. The differential in the case of basic education, however, was n^ligible 
for male youths (16.2 percent versus 15.4 percent), whereas it was small but noticeable 
for female youths. 
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The treatment group in this subgroup of male youths generally received a comparable 
number of hours of each service as their female youth counterparts, but the male youth 
control group in each case received a greater amount (number of hours) of each ser\'ice. 
As a result, the treatment-control group differences in the amounts of each service received 
(column 6) were somewhat smaller for male youths than for female youths. 

Note, also, that job search assistance was again a prominent service recei\'ed by this 
subgroup, with 30.8 percent of male youth treatment group members having received it. 

The O JT/JSA Subgroup. The male youth OJT/JSA subgroup was similar to the female 
youth OJT/JSA subgroup in the treatment group's likelihood of receiving on-the-job training, 
in the amount of OJT received, and in the treatment-control group differentials in both the 
likelihood and the amount of OJT receipt. Male* youth treatment group members in this 
subgroup were somewhat more likely than their female youth counterparts to receive job 
search assistance (32.2 percent versus 28.3 percent), the other key service in this strategy. 

The Other Services Subgroup. In this subgroup we find some differences in the t>pes 
of services received by male and female youths. Specifically, male youth treatment group 
members in the other services subgroup were somewhat more likely than their female 
counterparts to receive miscellaneous services (35.3 percent versus 28.5 percent) and less 
likely to receive basic education (13.5 percent versus 24.9 percent).*^ Nevertheless, the 
males' relatively small likelihood of receiving job search assistance as part of this service 
strategy was virtually the same as the females. And again the treatment-control group 
differences in the likelihood and amount of receipt of classroom training in occupational 
skills and basic education were small. 

Nevertheless, the males' relatively small likelihood of receiving job search assistance 
as part of this service strategy was virtually the same as the females'. And again, the 
treatment-control group differences in the likelihood and amount of receipt of classroom 
occupational skills training and basic education were small. 

Summary of Differences in Service Receipt. The preceding comparisons of service 
receipt across service strategies and between the treatment and control groups within each 
service strategy subgroup have yielded basic findings quite similar to those for female youths . 

Specifically, many treatment group members did not receive the service recommended 
for them, because of nonenroUment or receipt of another service. The three service strategy 



22. Again, the survey-based estimates of receipt of classroom training in occupational skills and 
basic education shown in exhibits 6.4 and 6.9 differ from tliose based on SDA data in Chapter 3 (Exhibit 
3.18), for reasons discussed earlier. 
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Exhibit 6.9 Receipt ofEmploymerU and Training Services: Male Youth Treatjnent Group and 
Control Group, by Service Strategy Subgroup 



Mean hours of service 
per sample mettiher 



Specific program 



service 



Percentage r eceiving service 

Treatment Control UtJJerence, ireatment Control UiJJerence. 

group in % pts. group group in hours 

a) (S) (^) 



Classroom training in 

occupational skills ' 
Basic education* * 
On-the-job training 

(JTPA only) 
Work experience 

(JTPA only) 
Job search assistance * 

(JTPA only) 
Miscellaneous^*** 

(JTPA only) 



group 
(J) 



Classroom training subgroup 



42.6% 
16.2 

4.4 

6.5 
30.8 
7.9 



22.4% 
15.4 

0.9 

1.9 



20.4% 
0.8 

3.5 

4.6 



321 
87 

20^ 

39^ 



193 
70 



13^ 



127 
16 

18' 

25' 



Classroom training in 

occupational skills" * 
Basic education * * 
On-the-job training** 

(JTPA only) 
Work experience ** 

(JTPA only) 
Job search assistance 

(JTPA only) 
Miscellaneous^*** 

(JTPA only) 



OJT/JSA subgroup 



15.6% 
5.6 

30.5 

4.2 
32.2 

6.8 



8.8% 
6.4 

0.8 

1.6 



6.8% 
-0.8 

29.7 

2.6 



106 
9 

131^ 
16^ 



48 
41 

3*^ 

6' 



58 
-32 

128'' 

10*^ 



(Continued) 
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Exliibit 6,9 Receipt of Eniploymeni and Training Services: Male Yourh Treamieni Croup atul 
Control Croup, by Service Strategy Subgroup (continued) 





Perceniag 


e receiving 


service 


Mean hours of service 
per sample mernber 






Treatmeni 


Control 


Difference, 


Treatmeni 


Control Difference, 


^pcClJlC P'Oj^r Uf li 






in % pis. 


group 


group in 


fiours 


service 


0) 


a) 


(S) 


(4) 


(5) 


(6) 








Other services subgroup 






Classroom iraining in 














occupational skills 


213% 


20.1% 


1.2% 


129 


119 


10 


Basic education 


13.5 


12.4 


1.1 


85 


86 


-1 


On-lhc-job training 








c 


< 




(JTPA only) 


3.9 


0.0 


3.9 


14 


0 


14 ' 


Work experience 








< 


c 




(JTPA only) 


3.2 


o.s 


2.4 


22 


14 




Job search assisuncc 














(JTPA only) 


12.0 












Miscellaneous 














(JTPA only) 


35.3 













Sources. 

+ Unadjuswd frequendet in this row ttt bMcd on Ftnt FoUowHip Survey dtu on ftocipi of Ibc icrvioe from my prxA-idcr. 
+ Unadjusted frequendet m ihii row mt bued oo euoUmBnt and ti«ckii^ dtu fiumUie 16 SDAs« theben cvaUablecUu 
on recdpi of ihit ictvice. Alihough the dju «e for JTPA TiUc U-A-fiindai aecviocft only, Uut lavicc U typically noi 
funded by non-JTPA providcn. 

+ + + UnadjusialffcqijaKsei in this row alto baaed 00 SOAeuoUi^ Cooaparable d«a oo roodpi of 

this aervioe from o<ha piovidcn «cre ooi avatlaUe; nor cocnpant4c dau oo loadpt by ootfiol group members. 
Notes: Saxtple »izc, classroom tnioing wbjroup: trciimcoi giotip « 354. oootrol froup « 172; QTT/JSA «ib5ro«p: iiratnw* 
gnx^ «4U.ooouolfn5up «204;oihcraeroceB«ibffOup:Uctoe*gto^ 176. Bocauc of missing 

dau, san^e sizes for sccvices calculKed fix>m differeoi dtU aouroes ovy vaiy. Tetts of «aa«ical sigoificaace were skot 
pcrformod for this exhibit. 

a. Lasting k>ager than ooe iMDck. 

b. Utting k>i^a than ooe ynodi. 'Basic education' includes Aduh Basic Educatioa (ABE), high school or Ocncnd 
Educ«ional Devdopincit (GED) picpantion, and English as a Second Lv^uagc (ESL). 

c. Hours, assuming a full-time job tt 40 houn per week. 

d. *Mjicd!«)eous* incli^les assetsmesu, job-readiness tnining, customized tnuning, vocational expl oral ion, job shadowing, 
and tjyout en^Ioyment, among other services. 
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subgroups do, however, represent distinctly different clusters of services actually reccived. 
As was true for female youths, most members of the male youth treatment group who w^re 
enrolled in JTPA received one or both of two key services, which varied by service strateg\* 
subgroup: classroom training in occupational skills and basic education for the classroom 
training subgroup; on-the-job training and job search assistance for the OJT/JSA subgroup; 
and basic education and miscellaneous services for the other services subgroup. The 
treatment-control group difiisrenccs in the average amoimt of each service received were 
modest. 

Thus, in terms of the types of services received across service strategy subgroups and 
the basic patterns of services received within each subgroup, the male youths were similar 
to the female youths. Minor gender differences in service receipt were apparent, however, 
in two of the three subgroups. In the classrocmi trainiiig subgroup treatment-control group 
differences in the amount of each service received were smaller among the males than among 
the females. And in the other services subgroup the males were somewhat more likely than 
the females to receive miscellaneous services and less likely to receive basic education. 

Impacts on High School Attainment As shown in the bottom p&nel of Exhibit 6.10, 
estimated program impacts on the attainment of a training^related high school credential 
were positive and statistically significant both for male youths overall (with a 6.0 percentage 
point impact) and for those who were high school dropouts upon their 2q>plication to JTPA 
(9.9 percentage points). The impacts were positive and statistically significant at between 
9.0 and iO.l percentage points for male youth high school dropouts in all three service 
strategy subgroups. A smaller proportion of the treatment group received a training-related 
high school credential in the OJT/JSA subgroup, however— that is, in the subgroup that 
placed the most emphasis on immediate employment. And the impact on the full sample 
in this subgroup was also smaller than on the other two subgroups. 

TTiese findings are similar to those for female youths, with two exceptions. For the 
OJT/JSA subgroup the finding for the female high school dropouts was small and statistically 
insignificant, while it was large and significant for male high school dropouts. And for 
the classroom training subgroup the estimated impact on high school dropouts was greater 
for the females than the males (16.4 percentage points versus 9.0 percentage points). 



Differences in Baseline Characteristics across Service 
Strategy Subgroups: JTPA Assignees 

Exhibit 6. 1 1 shows selected baseline characteristics of the out-of-school male youth assignees, 
overall and by service strategy subgroup. As was true of the fenude youths, the characteristics 
of each service strategy subgroup of male youths were quite different, again highlighting 
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Exhibit 6, 10 Impaas on Attainmsnt of a Training-ReUued High School Diploma or 
GED Certificate: Male Youth JTPA Assignees Overall and High School 
Dropout Subgroup, by Service Strategy Subgroup 



Percentage attaining a traitiing-relaied 
high school credensial 





Sample 


Coforol 


Difference. 




size* 


Assignees group 


in % pts. 




(1) 


(2) (3) 


(4) 






Classroom training subgroup 




Full sample 


509 


16.6% 10.3% 




High school dropouts 


302 


27.3 18.3 


9.0* 






OJT/JSA subgroup 




Full sample 


595 


5.8 2.0 


3.8**« 


High school dropouts 


236 


14.9 4.9 


10,1*** 






Other services subgroup 




Full sample 


588 


18.7 11.8 


6.9** 


High school dropouts 


417 


26.1 16.9 


9.1** 






AU subgroups 




Full sample 


1,692 


13.7 7.7 


6.0*** 


High school dropouts 


955 


2:^.9 14.0 


9.9*** 



Sources: Unadjured frequeocies based on Backgrouod Information Form reiponses «d Fine Fc»llow-up 
Surv^ reaponses. 

a Treatment and control groups combined. 

b. * Attainment of a training-related higb school credeotia!* is defmed as the combination of having 

received some school or training serice and having attained a high school diploma or Genera] Educational 
Development Cettificate at some time during the 18-nK>ath fcllow^up period. 

• Sutistically significant at the . 10 levdl, ** at the .05 level, ••* ac the .01 levd (two-tailed test). 



the fact that it is not possible to inteipret the impact findings by service strateg}' subgroup 
as indicating which service strategy would be most appropriate for the average male out- 
of-school youth. 

Again, the OJT/JSA subgroup was, by almost every measure, more employable that 
the other two subgroups; its members were more likely to have a high school credential, 
to have recent work experience, and to have been married and older. This conclusion is 
borne out by the estimated mean earnings for each subgroup over the 12 months preceding 
application to JTPA (the third main panel in Exhibit 6. 1 1): $2,742 for the OJT/JSA subgroup 
and only $1,779 and $1,671 for the other two subgroups, respectively. Unlike the females, 
however, there was not such a clear distinction between the other two subgroups of males, 
that is, with the classroom training subgroup tending to &11 between the OJT/JSA subgroup 
and the other services subgroup, as it did for female youths. Among male youths the other 
two subgroups tended to alternate in the middle position on different measures. 
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Exhibit 6.11 Selected Baseline Characteristics: Male Youth JTPA Assignees, by Service 
Strategy Subgroup 









OJT/ 


Other 




All 


trairutig 


JSA 


services 




subgroups 


subgroup 


subgroup 


subgroup 


Characteristic 


(1) 


a) 


(3) ~ 


(4) 


EUinidty 










White, non-Hiipuiic 


53.7% 


49.0% 


69.3% 


43.7% 


BItck, non-Hispanic 


29.5 


25.2 


17.2 


43.9 


HiipAoic 


14.7 


24.7 


12.3 


8.8 


American Indian or Aiadcan Native 


1.1 


0.5 


0.6 


2.1 


Asian or Pacific Islander 


1.0 


0.7 


0.6 


1.5 












Receiving cash welfare ' 


11.0% 


13.1% 


10.4% 


9.9% 


hi ch school dinlotna or 












59.1 


61.8 


41.2 


72.5 


^Unrlr^A Ia£c than 1") w^^c 










in naitf 1^ tnonthc 


47.0 


50.3 


35.5 


55.2 


Number of barriers 










None of the above 


23.9 


20.3 


37.3 


14.3 


One of the above 


40.9 


41.2 


42.3 


39.5 


Two of the above 


29.5 


32.4 


17.1 


38.8 


All three of the above 


5.7 


6.1 


3.3 


7.5 


Work And tnunmg histories 










Ever employed 


S3 .9% 


82.8% 


91.3% 


78.3% 


Mean individual earnings 










in pact 12 months 


$2,071 


$1,779 


$2,742 


$1,671 


Hourly earnings in most receiu job 










Never employed 


16.1% 


17.3% 


8.7% 


21.7% 


Leas than $4 


34.9 


37.8 


35.7 


32.0 


$4 or more 


49.0 


44.9 


55.6 


46.3 


Employed upon application 


11.4 


11.5 


11.7 


11.0 


Previously received occupational 










training 


29.7 


24.8 


32.0 


31.6 


Public atfisunce iUtl^s 










Receiving any public acsittance ^ 


29.5% 


32.9% 


27.6% 


28.3% 


Receiving AFDC 


6.2 


7.5 


4.7 


6.4 


Receiving food Mamps 


25.0 


27.3 


22.0 


25.7 


Receiving other public assiAance ^ 


11.0 


12.9 


12.3 


8.3 



4 i O 
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Exhibit 6.11 Selected Baseline Characteristics: Male Youth JTFA Assignees, by Service 
Strategy Subgroup (continued) 



Cliaract eristic 



All 
subgroups 
(1) 



Classroom 
training 
subgroup 

a) 



OJT/ Other 
JSA services 
subgroup subgroup 
(3) W 



AFDC history 

Never AFDC case head 

AFDC case head less than 2 yean 

AFDC case head 2 years or more 
JTPA required for welfare, food 

stamps, or WIN program 

Household composition 

No spx>use or own chUd present 

Own chUd any age, 
no spouse present 

Spouse present, with or 
without own child 
Family income in past 12 months 

< S3 .000 

S3 .000 - $6,000 

$6,001 - S9,000 

> S9.000 

Living in public housing 
Yes 
No 

Age at random assignment 
16-19 
20-21 
Mean 

Arrested since age 16 

Yes 

No 
Scunple size 



97.8% 
1.9 
0.2 

5.9% 
85.2% 

4.3 

10.5 

39.7% 
26.3 
11.3 
22.8 

10.7% 
89.3 

62.1% 
37.9 

1S.9 

24.7% 
75.3 
1,436 



97.9% 
2.1 
0.0 

6.5% 

84.6% 

6.0 

9.8 

38.7% 
27.2 
11.6 
22.6 

9.8% 
90.2 

69.0% 
31.0 

18.6 

21.4% 

78.6 

429 



96.8% 
3.2 
0.0 

5.8% 
82.2% 

2.8 

15.0 

35.6% 
26.7 
12.3 
25.3 

8.9% 
91.1 

51.7% 
48.3 

19.4 

25.6% 
74.4 
472 



98.7% 
0.7 
0.7 

5.5% 

89.1% 

3.0 

7.9 

25.2 
10.1 
20.6 

12.9% 
87.1 

65.8% 
34.2 

18.7 

26.6% 
73.4 

535 



Source: Unftdjusted frequencies b«ftod on Bftckground Infomution Form rcipoDtes. 

a. AFDC. General Aiiisuncc, or other welfare except food sumpi. 

b. 'Any public asiisuncc' iocludes the followinf •ourcea of atiiiunce: AFDC, food lUmp^, unemployment insurance, 
houiing atsiiianoe. and other cuh Miiitaoce. 

c. *Other public aisiiiance* includea uoemploymeat inturancc, bousinj aiiiitince, and other (noo- AFDC) cash assistance. 

d. WIN is the federal Woric Incentive protram. 

t. The perccDtafe arrested betweeo a(e 16 and random asii{iAnent is bas«d on responses 

to the First Follow-up Survey by treatment (roup memben who were includod in the analysis of impacts on earnings. 



2"4 
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As noted at the start of this chapter, by far the most striking differences bet>\'een male 
and female youths overall were in their rates of single parenting (4 percent for males versus 
36 percent for females) and their arrest rates (25 percent versus 6 percent).^ The two 
genders also differed on several other measures of their cmployability (sec columns 1 m 
exhibits 6. 1 1 and 6.6>— perh^s best summarized by the difference in their average earnings 
in the 12 months preceding their application to JTPA: $2,071 for the males vereus only 
$1,305 for the females. The later section detailing impacts on key subgroups will show, 
however, that it is the difference in arrest rates that accounts for most of the difference 
in program impacts between the two genders. 

But the arrest rates did not vary sufficiently among service strategy subgroups to account 
for the differences in average impacts among these subgroups that are presented in the next 
subsection. Specifically, among male youths 21.4 percent of the classroom training subgroup, 
25.6 percent of the OJT/JSA subgroup, and 26.6 percent of the other services subgroup 
reported a previous arrest. The corresponding arrest rates for female youths were 5,8 percent 
for classroom training, 8.5 percent for OJT/JSA, and 4.7 percent for other services (Exhibit 6.6), 



Impacts on Earnings: JTPA Assignees and Enrollees in 
Each Service Strategy Subgroup 

Exhibit 6.12, which shows the estimated impacts on earnings for each service strategy' 
subgroup of male youths, indicates that:^^ 

• The classroom training strategy yielded some opportunity costs iii the first 
quarter, possibly a small payoff in the second quarter, and then earnings close 
to what they would have been without JTPA in quarters 3 through 6. Over 
the follow-up period as a whole the estimated impact on earnings was $-259 
per assignee, or -2.6 percent of the control group mean of $9,783— a small 
loss that was not statistically significant. 

• The findings for the OJT/JSA strategy were very different. The estimated 
impacts on earnings here were negative throughout the foUow-iip period, for 

23. The arrest rates Dhovvn in Exhibit 6.11 for male youths and Exhibit 6.6 for female youths are 
based on responses to the First Follow-up Survey, not the Background Information Form which was the 
data source for aU the other baseline characteristics shown in those cxhibiu. The First Follow-up Survey 
did not obtain information on previous arrests among adults, however. 

24. Graphs of the monthly earnings trends on which Exhibit 6.12 is based appear as Exhibit H. 1 1 
in Appendix H- As noted earlier, the earnings and impact estimates include wages paid to JTPA participants 
in OJT positions. The program reimbursed employers about $600 per male youth CUT participant during 
the Ig-month follow-up period. Within the OJT/JSA subgroup the reimbursement was about $190 per 
male youth assignee over the follow-up period. 
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Exhibii 6, 12 Impacts on Earnings: Male Youth JTPA Assignees and 
EnroUees, by Service Strategy Subgroup 





Corarol 


Impact per assignee 


Inferred impact per 




mean 


In 5 


As % qf(l) 


enrolUe, in $ 


Period 


(1) 


(2) 


(3) 








Classroom training subgroup 




QuMter 1 


% 1.226 


$ -300** 


-24.5% 


% -441 


2 


1,345 


96 


7.2 


141 


3 


1,655 


•2 


-0.1 


-3 


4 


1,773 


0 


0.0 


-1 




1,889 


.56 


-2.9 


-81 


6 


1,895 


4 


0.2 


s 


All quaiters 


9,783 


-259 


-2.6 


-380 






OJT/JSA subgroup 




Quarter 1 


$ 1,651 


$ -57 


-3.4% 


$ -103 


2 


1,988 


-219 


-11.0 


-398 


3 


2,197 


-302* 


-13.8 


-552 


4 


2,160 


•203 


-9.4 


-371 


S 


2,316 


-192 


-8.3 


-351 


6 


2,452 


-339** 


-13.8 


-617 


All quarters 


12,765 


-1,313* 


-10.3 


-2,392 






Other services subgroup 




Quarter 1 


$ 1,362 


$ -285** 


-20.9% 


$ -432 


2 


1,457 


-121 


-8.3 


-182 


3 


1,605 


■ -218 


-13.6 


-330 


4 


1,751 


-276* 


-15.7 


-417 


S 


1,766 


-114 


-6.4 


-172 


6 


1,899 


-292** 


-15.4 


-442 


All quarters 


9,839 


-1,305* 


-13.3 


-1,976 



Source: E«tiraaiM bMed on Hnt Follow-up Survey rMpooM*. 

Note*: Sample size, clMftroom tnuning nibfroi^: MtifDeei » 354, control group » 172; OJT/JSA wbgroup: 
Autgneet » 411, control group » 204; other lervioe* wbgroup: Mitgoeei » 431, control group « 176. 
E«timtte« ire regteMtOft-edjutted to control for difTerenoM in bafteline cheracter^^tici bctwwn the 
treatment group md control group; aee Appendix D. 

* Statistically ugmficant at the .10 level, at the .05 level, at the .01 level (two-tailed teat). 
Significance levdt for column 3 are identical *o thoae in column 2. Teau of etatiitical aignificaooe were 
aot performed for column 4. 
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a total earnings impact of $-1,3 13 per assignee, or -10.3 percent of the control 
group mean — a loss that was both large and statistically significant. Note that 
the control group mean was much higher for the OJT/JSA subgroup — at 
$ 12,765— than that for the other two subgroups, at around $9,800. This finding 
suggests that, as was true of adults and female youths, the male youths with 
the greatest potential earnings were recommended for the OJT/JSA strategy. 
In this case, however, the strategy did not serve well the male youths deemed 
appropriate for it. 

• The impact findings for the other services strategy were similar to those for 
the OJT/JSA strategy, with negative estimated impacts throughout the follow- 
up period, and a negative impact for the follow-up period as a whole — $- 1 ,305 — 
that was abnost identical to that for the OJT/JSA subgroup. 



Impacts on the Components of Earnings: JTPA Assignees in 
Each Service Strategy Subgroup 

Exhibit 6.13 presents estimates of program impacts on the four components of the earnings 
impacts, displayed in columns 2 through 5.^ 

In the first panel the negative impacts in the first quarter are again consistent with the 
opportunity costs associated with the in-program period in the classroom training subgroup. 
There was a small gain (2.7 percent) in the percentage employed at some time during the 
follow-up period as a whole (column 2), but that was more than offect by the -5.4 percent 
drop in weeks worked per worker— to yield the -2.6 percent loss in earnings per assignee. 
And despix the goal of classroom occupational training to increase job-related skills, the 
estimated impacts on earnings per hour worked by those who worked (column 5) were small 
and showed no consistent trend over time. 

The most striking finding in the second panel, for the OJT/JSA subgroup, is that the 
program had a negative impact on employment rates in every follow-up quarter except the 
first, yet the impact on employment for the follow-up period as a whole was actually positive, 
at 1.0 percent (although this estimate was not statistically significant). In other words, 
treatment group members were less likely than control group members to be employed in 
any -^iven quarter, but they were slightly more likely to be employed at some time during 
the follow-up period. This suggests that treatment group members obtained their jobs later, 
held them for a shorter period, or both. 



25. For the corresponding estimated impacts on male youths overall, see Exhibit H.IO in Appendix 
H. Exhibit R9 presents impacts on the percentage employed and the number of weeks and hours wortced 
for all male youths, including those who were never employed during the follow-up period. 
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Exhibit 6,13 Percentage Impacts on Earnings and Its Components: Male Youth 
JTPA Assignees, by Service Strategy Subgroup 









Workers 


Weeks Hours worked 


Earnings 






Earnings 


per 


worked 


per week 


per hour 






per assignee 


assignee 


per worker 


worked 


Marked 


Period 






a) 


(3) 




(5) 








Classroom training subgroup 






Quarter 


1 


-24.5% 


-11.6% 


-10.0% 


-3.9% 


-1.3% 




2 


7.2 


6.2 


-3.8 




1.7 




3 




3.9 


-2.6 


-0.2 


-1.1 




4 


0.0 


0.4 


-0.4 


0.7 


-0.7 




5 


-2.9 


^.3 


-0.1 


-2.2 


-0.4 




6 


0.2 


-2.3 


-0.4 


0.7 


2.2 


All quarters 


-2.6 


2.7 


-5.4 


0 0 

v.v 


0.3 










OJT/JSA subgroup 






Quarter 


1 


-3.45t 


2.5% 


1.4% 


-1.8% 


-5.4% 




2 


-11.0 


-5.6 


^).6 


-0.9 


-4.3 




3 


-13.8 


• ^.5 


-2.7 


-5.0 


-0.3 




4 


-9.4 


-7.1 


4.2 


-6.4 


0.0 




5 


-8.3 


-3.0 


-0.7 


-4.5 


-0.3 




6 


-13.8 


-5.3 


-1.9 


.4.3 


-3.0 


All quarters 


-10.3 


1.0 


-5.5 


^.0 


-2.1 










Other services subgroup 






Quarter 


1 


-20.956 


-13.3% 


-2.6% 


-5.8% 


-0.5% 




2 


-8.3 


-0.3 


-2.6 


-3.8 


-1.8 




3 


-13.6 


-3.4 


-4.9 


-3.8 


-2.2 




4 


-15.7 


-€.4 


-5.6 


-2.4 


-2.4 




5 


^.4 


-3.5 


5.1 


-0.4 


-7.5 




6 


-15.4 


-0.4 


-5.7 


-1.9 


-8.2 


AU quarters 


-13.3 


1.9 


-8.8 


-2.8 


-4.0 



Source: Eaiimafcrs baaed on Fim Follow-up Suivcy itipoaaea. 

Notes: Sample wee, cUmtodb tnintflf wbf loup: auisoees « 354, oooftrol group » 172; Orr/ISA wbf raup: 
assfoeei « 4U» cooCrol (roup » 204; other aeivicea aibgioup: amgaeet » 431, ooolrol f roup » 176. E^iwiiet 
ife ref reatioft-adjuaed to cosirol for diffeitacea in baieliae chanctetiftica becweea the tmtaeai group aad 
coiurol group; tee Appendix D. Columos 2 through 5 diiptty the inpACt as a petceaiage of the conapoodiag 
coDtrol mtm (act shown). For column 2 this meauc the impact on the employment nte is calculated as a 
percentage of the oMan nte for the control group. Te«s of statistical significance wfsc not perfonned for any of 
the columns in this exhibit. 
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Evidence presented in Exhibit H.23 of Appendix H on the amount of time betvs een 
random assignment and the first job obtained by youths in the sample suggests that the 
OJT/JS A service strategy may have slightly delayed employment for male youths (although 
the impact estimate was not statistically significant). Our final report wU present a further 
analysis of the timing and duration of job spells for members of each service stratcgj* 
subgroup. 

A final point to consider is that most of the program induced earnings loss experienced 
by male youths in the OJT/JS A subgroup was due to the feet that treatment group members 
worked fewer weeks than control group members did during the 18-month follow-up period 
and treatment group members worked fewer hours per week cnq>loyed. 

Returning to Exhibit 6.13, we find a similar pattern for the other services subgroup, 
although the quartcr-by-quarter impacts on the percentage employed tend to more closely 
parallel those for the classroom training subgroup. There was a sharp initial decline in 
employment in the first quarter (consistent with a short investment of time in the program) 
and continued negative employment effects through the remaining five quarters. The 
estimated effects on weeks and hours woikcd for those employed were negative throughout 
the follow-up period, as were the estimated effects on earnings per hour worked, especially 
in the last two quarters. And although the estimated impacts on quarterly employment 
rates were negative, the overall impact on the employment rate for the follow-up period 
as a whole was positive (though not statistically significant). In all three service sirategy 
subgroups the estimated negative impact on weeks worked per worker was the most salient 
component of the overall impact on earnings. 



Impacts on Earnings: Female and Male Youths in 
Selected Key Subgroups 

This section presents estimates of JTPA impacts on the 18-month earnings of selected key 
subgroups of youths. The majority of the key subgroups of youths arc defined in the same 
way as were the key subgroups of adult women and men (exhibits 4.15 and 5.14); but 
we have included two additional sets of subgroups to examine potential differences in program 
impacts with respect to characteristics that we expected to be more relevant for youths 
than for adults, namely, previous occupational training and reported arrests.^ 

The analysis attempts to identify subgroups for which the JTPA Title II-A program 
was effective in the 16 study sites and subgroups for which the program was not efifcctive. 

26. Wc selected the key subgroups examined in chapters 4.5. ind 6 based on their relevance to 
pohcy discussions, before wc calculated the estimates. In other words, wc did not select them on the 
basis of the size or significance of the program impact estimates. 



27^ 
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Knowledge of the fonner groups &cilitate future research on the factors that lead to 
program success, wiiile knowledge of the later can help target efforts to improve the program . 
But the analysis by itself cannot determine why the program was woildng for some groups 
and not for others. Nor does it lead to simple prescriptions about how to in^rovc the program. 
In short, it can only measure the effects of the program, the way it actually operated, on 
the people it actually served. 

Exhibits 6.14 and 6.15 present the findings for female and male youths in the same 
way that corresponding findings were presented in chapters 4 and 5. Each panel in the 
exhibits defines a set of subgroups in terms of a particular set of baseline characteristics. 
For example, the first panel defines subgroups according to the ethnicity of sample members; 
the second, according to specific employment barriers faced by sample members; and the 
third, according to their work histories. 

The first column in the exhibits presents the sample size for each subgroup, including 
both treatment group and control group members. The second column presents the mean 
earnings of control group members over the 18-*month follow**up period. Column 3 displays 
estimates of the average program impacts on total l$*month eamiiigs for each subgroup. 
Asterisks beside the impact estimates denote that the estimates are statistically significantly 
different from zero, whereas asterisks in the final row in each panel indicate that impact 
estimates in the panel are statistically significantly different fi-om one another. 

The fourth column in the exhibits displays the column 3 estimates adjusted for differences 
in the distributions of the subgroups across the 16 study sites. These estimates control 
statistically for the extent to which some subgroups were more heavily concentrated in sites 
with more positive or negative impacts than the other subgroups in the panel. Likewise 
the fifth column displays the colunui 3 estimates adjusted both for differences in subgroup 
distributions across sites and for differences in subgroup distributions across service 
strategies.^ 

To interpret the findings in the exhibit one should proceed as follows. First, to assess 
the likely impact for any given subgroup, examine the size and significance of the impact 
estimates in column 3. Estimates that are statistically significant are those that arc most 
likely to represent true impacts for a subgroup, ais opposed to chance results due to random 
sampling error. One should place the most confidence in these significant estimates. The 
significance of the variation in impacts among subgroups is measured by an F-lest, with 
results reported in the last row of each panel. Those panels that contain subgroup impact 



27. The estimates in column. 4and5ofexhibits6.14and6.15adjustthedistributionofeachsubgroup 
to equal the distribution of female or male youths overall across sites (column 4) or across aites and service 
strategies (column 5). For a full description of the methodology for deriving these estimates, see 
Appendix D. 
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Exhibii 6. 14 Impacts on Total 18-Month Earnings: 
Subgroup 



FemaU Youth JTPA Assignees, by Selected Key 



Key subgrot^, 
iUfined by: 



SatnpU 
size* 



Control 
mean 



Impact, in $, adjusted pr 
sample Astribunon across: 

Sites and senice 
Impact, in $ Sites strategies 



/1 1 


(*/ 


(3) 


(4) 


(S) 


1.148 $ 


7,076 $ 


.122 


$ .348 $ 


.231 


749 


5,601 


-135 


455 


518 


366 


5,019 


.554 


.338 


.147 






s.t. 


n.t. 


B.S. 


701 


4497 


.391 


.515 


.393 


1,412 


7,1/4 






.287 






m.t. 




B.t. 


1,047 


4,192 


23 


42 


223 


1,146 


8,055 


-437 


.346 


.232 






n.s. 




B.S. 






-31 


.34 


109 


829 


8,886 


.255 




10ft 






ft.S< 




B*S« 


545 


9,964 


-260 


.303 


.278 


790 


6,552 


.236 


.265 


.149 


675 


4,486 


-451 


-496 


-353 


281 


2,189 


659 


716 


934 










B.S. 


514 


3,201 


.232 


1 


192 


1,073 


6,447 


.83 


.108 


-36 


713 


8,030 


.277 


.235 


.111 






a.s. 


tt.S. 


B.S. 


340 


9,794 


-218 


.351 


-178 


1,950 


5,595 


-172 


.93 


5 








B.S. 


B.S. 


582 


7,210 


-44: 


-301 


.274 


1,670 


5,838 


-83 


.169 


-15 








B.S. 


B.S. 



Ethnicity 

White, Qon-HiipBnic 
Blick, non-Hiiptnic 
Hisptnic 
V'UA, diffeiCBce laio&c subgroups 

Barriers to employment (in italic) 
Receiving cash welfare * 
No cash wdftre^ 

IMe«, differeace betweea nibgroups 
No high sdiool diploma or 
CED certificate 
Hi^ school diploma or 
GED cmificate 

F.te«, differeoce bctweea wbgroups 
Worked less then 13 weeks 
inpasx 12 months 
Worked 13 weeks or more 
in past 12 months 
F'tesi, differeace betweea wbgroups 

Number of barriers 
None of the above 
One of the above 
Two of the above 
AU three of the above 
F-iesi, differeace aaoBg nibgroups 

WoT-k history 
Never employed 
Earned < $4 hourly in latt job 
Earned $4 or > hourly in laic job 

F'leii, differeace amoag subgroups 
Employed upon application 
Not employed upon application 

F'tesi, differeace betweea subgroups 

PrevkMis ocaipntioiul training 
Yes 

No 

F-lc«, differcocebetweca subgroups 



(CauiMued) 
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Exhibit 6.14 Impacts on Total IS-Moitlh Earnings: FemaU Y mh JTPA Assignees, by Selected Key 
Subgrotip (continued) 



impact, in S, adjusted for 
sample disrribsaion across: 
Sample Control Unadjusted Sites and service 

Key subgroup, size * mem intact, in $ Sixes strategies 

defined by: (1) Q) (S) (4) (5) 



AFDC history 












Never AFDC cate besd 


1,634 


$ 6,656 


% -232 


$ -248 


$ -141 


AFDC caie bead less than 2 years 


487 


5080 


269 


349 


503 


AFDC case bead 2 years or more 


162 


4,125 


*1,089 


-1,028 


-984 


Fnett, differeaoe ainoo^ mibf roups 






a.s. 


a.s. 


n.s. 


JTPA required for welfare, food 
stamps, or WIN program' 






















Yes 


163 


2,994 


342 


193 


234 


No 


2,029 


6,518 


-204 


-249 


-143 


F-teA, diffcre&ce between subgroupt 






a.s. 


a.s. 


a.s. 


Housdiold compoiitioo 












No ipouse or owa child present 


1»111 


6,979 


-327 


-308 


-225 


Own child any age. 












no Ipouse present 


7S9 


5,796 


-380 


-272 


-149 


Spouse present, with or 












without own child 


252 


4,819 


1,007 


917 


1,029 


FhcA, differeace amoaf aibf roups 






a.s. 


a.s. 


a.s. 


Family income in past \2 mooths 












$6,000 or less 


1,524 


6,011 


-487 


-536 


-437 


More than $6,000 


557 


7,160 


511 


419 


519 


F^ec, differeace betw«ea subgroups 






a.i. 


a.s. 


a.s. 


Living in public housing 












Yes 


307 


4,077 


246 


238 


292 


No 


1,939 


6,602 


-288 


-340 


-249 


FHe«, differeace bcMveen subgroups 






a.s. 


a.s. 


a.s. 


Age at random assignment 












16-19 


1,400 


5,732 


-124 


-101 


19 


20-21 


900 


6,999 


-279 


-219 


-114 


FneA, differeoce between subgroups 






a.s. 


a.s. 


a.s. 


Arrested since age 1€ 












Yes 


125 


5,827 


705 


689 


74! 


No 


2,122 


6,251 


-200 


-154 


-19 


Fneft, differeoce betwees subgroups 






a.s. 


a.s. 


B.S. 


Recoomeoded for JS A only 












Yes 


49 


7366 


1321 


1,071 


n/a 


No 


2,251 


6,174 


-209 


-206 


n/a 


F-ieM, differeoce betweea subgroups 






a.s. 


a.s. 


a.s. 


FuU sample 


2,300 


6,225 


-182 


-182 


-182 



Source: Fgi— ift b«ie4 oo Fim Foll9v«up Survey teapoMct. 

Noiei: Fai—ift we teg feMtoft-aijuiiBil le ike ooairal for 4ifrefe»oet \% t«aeltae ctefMeriflict MiwtM ite UMMoeH graup Mtf coiirat. 
group: «t Appetdix D. Cootrol wtm» ft sot legieMioB^juiiad. Suipic wm for MMlly cxduiive whgwup' wiikla s pt^l ao 
woL KOTMvUy fu» to Ike napk tis for ike Mrget group ■• s wtek, bocouie pcneu U «ai«e4 MbgRMpt or whk aiMiit a«« oo ite 
voriabk und to itftoe ikt wbgroup ore cxduM. 
o. Tieaaeii oad ooouo) group* ooabiied. 

b. AFDC, Grienl Ammuoc, or oiher weUore uiopi food ■■■pi. 

c. WXK it Ike federal Work laoeative pragris. 

* Sauiniodly agMftoMt m ike .10 kvcl, h ike .05 level, si ike .01 kevcl (F^oit or iwoHoUed t-MiO; *».•.* mm i^ F*iea for tke 
difTcreBfle is iapoou bei««ee or tao^ ike fukgroupi is ike oaegoiy it mx ■aiakally ligsiftoMt. 



ERIC 



2^2 



220 • JTPA IS-MOhTTH IMPACTS / OUT-OF-SCHOOL YOUTHS 



estimates that are statistically significantly different from one another provide the strongest 
evidence that the true impacts for these subgroups were actually different. If the subgroup 
impact estimates within a panel are not statistically significantly different fi-om one another, 
the feet that their point estimates dififer does not provide strong evidence that the true impacts 
for the subgroups were actually different.^ 

Having examined the column 3 estimates for a set of subgroups in a panel, one should 
next read across the rows to their counterparts in the last two columns. To the extent 
that the variation in impaa estimates across subgroups changes as one moves fi'om the 
column 3 estimates to the estimates in columns 4 and 5, then effects related to sites or 
sites and service strategies explain part of the subgroup variation. In other words, the feet 
that one or more of the subgroups were concentrated in different sites or recommended 
for different service strategies explains part of the variation in estimated impacts. But if 
the variation in impact estimates shown in column 3 remains unchanged as one moves to 
the corresponding estimates in coltmms 4 and 5, differences in the distributions of the 
subgroups across sites or across sites and service strategies do not explain the variation 
in impacts among subgroups. 

The following discussion of the findings in exhibits 6.14 and 6.15 focuses primarily 
on subgroup impaa estimates that are statistically significantly different from zero and 
statistically significantly different from one another, because one can place the most 
confidence in these estimates. We begin each subsection, however, by discussing one set 
of subgroup estimates that are not statistically significantly different from one another — 
those for ethnic groups — because this lack of a difference is an important finding to note. 



Female Youths in Key Subgroups 

The top panel in Exhibit 6. 14 shows that program impact estimates for the three main ethnic 
groups of female youths — whites, blacks, and Hispanics — were neither statistically sig- 
nificant nor significantly different from one another. Thus, there no evidence that the Title 
II-A programs studied produced different impacts on the 18-month earnings of these 
subgroups. 



28. All the subgroup impact estimates in cxhibiu 6.14 and 6.15 arc based on ordinary least squares 
regressions on a pooled sample of all female youths and male youths» respectively* with the treatment 
indicator intemcted with the defming characteristic of the subgroup and (as appropriate) site or site and 
service strategy. This approach allowed direct calculation of the F*test for diflerences in impacts among 
subgroups in each panel of the exhibits. Subgroup impacts were also estimated on samples conuining 
only the subgroup of interest (not shown); in general, these "split file" estimates differed little from those 
based on the pooled regressions. 
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Indeed, although most of the estimates in Exhibit 6.14 were slightly to moderately 
negative (a few were positive), none was statistically significant. Moreover, within panels 
the impact estimates for each set of subgroups were not significantly different fi-om one 
another. The overall pattern of estimates in the exhibit therefore suggests that the JTPA 
program had little or no impaa on any of the subgroups of female youths identified for 
this analysis. 

Male Youths in Key Subgroups 

The top panel of the next exhibit, 6.15, shows large negative estimated program impacts 
on the earnings of white and Hispanic male youths, axid the estimate for white male youths 
was statistically significant. In contrast, the estimated impact on black male youths is close 
to zero. Despite these apparent differences among the main ethnic groups, however, the 
estimates were not statistically significantly different firom one another. These fimdings 
mean we have confidence that the estimate for white male youths reflects a real program 
impact on that subgroup, but we cannot be sure whether the impacts for blacks and Hispanics 
were different fi'om that for v^diites or were instead due to random sampling error. The 
column 3 estimates for blacks and especially Hispanics did change, when adjusted for the 
distributions of the two subgroups across sites (column 4) or sites and service strategies 
(column 5), but in neither case were the differences among ethnic groups statistically 
significant. There is no evidence, therefore that, JTPA produced systematically different 
impacts for the three main ethic groups of male youths. 

The remaining panels in Exhibit 6. 15 present a pattern that is quite different fi'om those 
for subgroups of female youths in the previous exhibit. Many subgroups of male youths 
had large, negative — and statistically significant — impact estimates; only 4 of 35 subgroup 
estimates in column 3 were positive, and none of these are statistically significant. 

But only two sets of male youth subgroups had findings within the set that were 
significantly different fi^om each other: the subgroups of male youths recommended for 
job search assistance only versus those not recommended only for this service (the last 
panel); and the subgroups of male youths who had been arrested between their sixteenth 
birthday and the time they sy)plied to JTPA versus those who had not been arrested. 

Because only 77 treatment and control group members (forming less that 5 percent 
of the male youth sample overall) were recommended for job search assistance (ISA) only, 
it is difficult to know what to make of the estimated S5,402 program-induced earnings 
loss for this subgroup. Furthermore, because of its small size, removing the subgroup 
fi'om the male youth sample did not seriously alter the impact estimate for the larger group. 
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Exhibit 6. 15 Impacts on Total IS-Monih Earnings: Male Youth JTFA Assigntis, by Selected Key- 
Subgroup 



Iwpaa, in adjusted for 



Key subgroup, 
defined by: 



Sample 
sizef 

0) 


Control 
mean 

(2) 


Jnpaet, in S 
(3) 


sample distribution across: 

Sites and service 
Sites strategies 
(4) (S) 


946 


$ 12,550 


$-1333" 


$-1,680" 


$-1,995"* 


522 


8,164 


75 


-506 


-414 


248 


10,126 




-21 










B.S. 


B.t. 


185 


8,815 


-56 


-357 


-578 


1,374 


11,292 


-1,020" 


.1,060* 








B.S. 


B.S. 


B.t. 




10,087 


.1,144« 


-1,471" 


-1,454** 




1 1 A\0 












Bat. 


B.S. 


B.t. 


734 


•»olo 








842 


12,808 


.832 


-829 


-1,011 






B.t. 


B.S. 


B.t. 


475 






-324 


-506 


733 


10,810 


.69S 


-765 


-839 


455 


8,520 


-1,242 


-1,481 


-1,531* 


81 


7,642 


.1,278 


.1,600 


-1,743 






B.S. 


B.t. 


B.t. 


269 


7,858 


-1,067 


-1,233 


-1,249 


617 


9,687 


.745 


-969 


-1,095 


862 


12,435 


.806 


.842 


-943 






B.S. 


B.t. 


B.t. 


213 


11488 


526 


266 


163 


1,521 


10,722 


.1,073** 


-1,213" 


.1,329** 






B.S. 


B.t. 


B.t. 


527 


11,903 


-575 


-585 


-769 


1,183 


10,285 


-904* 


-1,049 


-1,118* 






B.S. 


B.t. 


B.t. 



EOmidtj 

White, ooD-Hiipaiuc 
BiBck, iKm-Hi^>Bnic 
HiapBnic 
F-UA, diflfctcoce UBoaf aibg roupt 

BBrricn to cmployuMol Gn italic) 
Receiwtg cask welfare * 
No CBsh wdfve^ 

F-ie«, difratiic< betweeo wbtroupt 
So high school diploma or 
C^D certificate 
High school diploma or 
GED ccctxftcBte 

F-Cc«^ differeoce betweeo wbf roupt 
Worked less Otan IS weeks 
in past 12 months 
Wortced 13 weeks or more 
in pMt 12 months 
F-ie«, diflfeteace between wbf roups 

Number of barriers 
None of the above 
One of the above 
Two of the above 
AU three of the above 
F-teit, difference amonf mibfroups 

Work history 
Never employed 
Eamed < $4 bouriy in laat job 
Earned $4 or > hourly in la« job 

F-teit, diffeitace amon^ mibf roups 
Employed upon application 
Not employed upon application 

F-ieat, difference between mibf roups 

Previous occupAtioiial trmniog 

Yrs 
No 

F4eA, difference between aibfroups 



(Omdmrnt) 
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Exhibil 6.25 Impacts on Total IS-Monxh Earnings: Male Youxh JTPA Assignees, by Selected Key 
Subgroup (continued) 



Jnrpact, in S, adjusted for 
sanple distribution across: 





Sanple 


Control 




Sites md service 


Key subgroup. 


size • 


mean 


Impact, in S 


Stes 


strategies 


defined by: 


0) 


(2) 


(3) 


(4) 


(S) 


JTPA required for welfare, food 












sUunps, or WIN program 










$ 542 


Y« 


91 


S 7,178 


% 462 


% 552 


No 


1»552 


10,8S3 


-•14" 


-947 


-1,052* 


F-ce«i, diflienace becwees wbgroups 






a.t. 


a.t. 


a.s. 


Household cooipofitioa 












No spouse or own chud present 


1,411 


10,679 


-1,176** 


-1,238** 


-1,375** 


Own diild Any s^e. 












no spouse, present 




in fnc\ 
11/ ,o /u 




-2,641. 


-2,762 


Spouse present, with or 












WluK/Ui vWu CIUlU 


181 
to I 




1^22 


1,OOU 


1 *7^Q 


Pja^ differ^ncA »mmk» MibfirouOS 






a.s. 




# 


FtmUy incoise in past 12 mooUis 










-1,076 


$6,000 or less 


1,038 


10,495 


-568 


-978 


More loan «o,uai 








*t83 


-948 


F-teM, diffemce becweea tubptMips 






ft.t. 


a.s. 


m.s. 


Living in public bousing 












Yes 


188 


9,564 


-956 


-1,185 


-1,298 


No 


1^ 


10,905 


-722 


-759 


-852 


F4est, differeace between subgroups 






a.t. 


a.s. 


a.s. 


Age at random •«*ignm»nf 












16-19 


1,053 


10,464 


-1,314»» 


-1,795»»» 


-1,881*** 


20-21 


695 


11,269 


-179 


-177 


-288 


F-test, differeace between subgroups 






a.s. 


# 


# 


Arrested ance age 16 












Yes 


401 


11037 


-3,C38»»» 


-3,150»»» 


-3,254*** 


No 


1,313 


10,696 


-22-; 


-219 


-342 


F4ett, differeace between subgroups 












Recommended for JSA ooly 












Yes 


77 


15^534 


-5,402»* 


-6,297»» 


n/a 


No 


1,671 


10,649 


-687 


-801 


tt/a 


F4est, differeace betweea subgroups 












Full sample 


1,748 


10,736 


-854»» 


-854»» 


-854** 



Sourae: EitiMai batad m Pint FoUow*4ip Survey imt ioMii. 

NoCa: Betkmlm arc rBgrBMioo-adiMiiad te oodiol fof difftii^m m fcirlinf chii»cUriadoi bctww the » ■aiM m gtwy 

and oo^rol gimp; tat Apfmd'a. D. Co^iol meana aic Ml ftcrMioa<adjuM«d. Saopte sizai for wlually evkiMve ai^groupt 

witha » paad do act unraiarily w to •aayte atse for Ihelaifci group a» « wkotc. baoauae pctaoaa m anittaj adbgrwpt 

or with aiiMit 4^ «• variabk uaad to daAac te auhgm^ w «xcludad. 

«. Tiaamifif sad oeMrot peufa caBbiia4. 

^. AFDC. Gcaeral AMiAaaae. cr eter wtUare exoipc food tauBfi. 

c. WIN U te Mml Woffc laoettfve progima. 

• SMiaioally afiiffoaM a te .10 kcval, « te .05 k^U a te .01 k«et (F-ieft or two^ikd *&.•.* Baaai te F-ie« for te 
diffaiuaa is tepacu ta^iaia «r aaaag te «bgr«up« a te caMgory U aoi MatiakaUy agaiAcMi. 
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reducing the $-854 average earnings loss for aU male youths to a still-large (but statistically 
insignificant) $-687 earnings loss for those male youths who were not recommended for 
ISA only (results shown in the exhibit). 

The impact estimate for the subgroup of male youths with a prior arrest (401 treatment 
and control group members) accounts, however, for a major portion of the estimated impact 
on the male youth sample. Removing this subgroup firom the sample reduced the significant 
$-854 average earnings loss for all male youths to a statistically insignificant $-224 earnings 
loss for male youths without a previous arrest. The estimated impact for previous arrestees 
was a highly significant $-3,038 earnings loss, as shown in column 3— a result that was 
significantly different, at the .01 level, firom the estimate for the nonarrested subgroup. In 
short, the 25 percent of the male youths who had been arrested before their JTPA application 
accounted for about 82 percent of the total program-induced earnings loss estimated for 
male youths overall.^ 

Our forthcoming final report will explore fiirther the potential sources and implications 
of this striking resuU. The limited analyses we have conducted to date (not shown here) 
suggest the following: 

• Male youths with previous arrest experienced large, negative program impacts 
on their earnings in all six follow^ ^p quarters. These estimates were statistically 
significant in five of the six follow-up quarters. Since the median duration 
of enrollments for this subgroup was relatively short (3.3 months), little of this 
earnings loss is likely to have been a result of time spent in training. 

• The impact estimates were also large and negative for all three service strategy 
subgroups of previous arrestees, at $-3,420 for the classroom training subgroup, 
$-5,746 for the OJT/JS A subgroup, and $-2,200 for the other services subgroup. 
The estimates for the classroom training and OJT/JSA subgroups were sta- 
tistically significant, and that for the other services subgroup was significant 
at a near-conventional level (.l^V 

• The impact estimates for this subgroup were also negative ml 3 of the 15 study 
sites, ^ although because of the small sample sizes involved these estimates were 
not statistically significant. As a result, the overall earnings loss for the previous 

29. The 82 percent figure represents the total prognm-induccd earnings loss for male youths with 
a previous aircst expressed as a percentage of the toul earnings loss for all male youths. The total earnings 
loss for each of these groups was computed as the product of the average earnings loss per Ucatmcnt 
group member times the number of treatment group members within the group. 

30. The Oakland site excluded youUis from the experiment; thus, there are 15 instead of 16 study 
sites for youths in the National JTPA Study. 
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arrestees is unlikely to be merely the results of the idiosyncrasies of only ^ 
few sites. 

• The types of jobs the previous arrestees reported during the 18-month period 
were mainly low-wage positions in service industries such as fast foods and 
maintenance and repair. These jobs are plausible types of jobs forthis subgroup, 
which suggests that the previous arrestees' response to the First Follow-up 
Survey (the data source for the impact analysis in this report) were not 
exaggerated. 

• Most of the observed earnings loss for the previous sutestees remained when 
those with the /i/^/2e5/ea/7};>7g5(the**outliers'') were removedfrom the analysis. 
The impact findings for this subgroup therefore represent more than extreme 
results for a few sample members. 

But despite the apparent consistency of all these estimates for previous arrestees within 
the male youth sample, our preliminary analyses also uncovered an important reason for 
exercising caution in interpreting these estimates. In particular, we have determined that 
these estimates, which are based on sanq)le members ' respcHise to the First Follow-up Survey, 
differ systematically and significantly from impact based on an alternative data source: 
earnings fi'om state unemployment insurance (UI) agencies. 

Appendix E presents a detailed comparison of estimates based on the two data sources — 
including estimates of earnings, employment, and program impacts on both earnings and 
employment — for all four target groups. Although the survey-based estimates differ 
some^^t fi'om the Ul-based estimates for all four target groups, the two sets of estimates 
differ in a similar way for both the treatment group aiul the control group — except in the 
case of the 401 male youths with a previous^a,rrest. 

In other words, although the Ul-based estimates of sample members' earnings differ 
from the survey-based estimates in absolute magnitude, the Ul-based and survey-based 
estimates of program impacts did not differ greatly for adult women, adult men, or female 
youths overall, or for male youths who reported never having been arrested. But the Ul- 
based estimated impact on the earnings of male youths with a previous arrest is statistically 
significantly different from the survey-based estimate. Specifically, the Ul-based impact 
estimate is a statistically insignificant S34, w4)ereasjthe survey-based estimate, as we have 
seen, is a statistically significant $-3,038. 

At present we do not fully understand the reasons for these ambiguous findings; our 
forthcoming final report will include further analysis that should supply us with a better 
understanding. In the meantime, then, the survey-based finding of a large, negative, and 
statistically significant program impact on the earnings of the previous arrestees among 
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male youths must be interpreted with caution. And even more important, because most 
of the negative impact estimated for male youths overall is due to the extremely large negati\e 
impact on the subgroup of male youths with a previous arrest, our findings for male \ ouths 
overall must also be interpreted with caution. 

At the same time, however, neither the survey-based nor the Ul-based estimates indicate 
a program-induced earnings gain for male youths. Thus, at the very least we can conclude 
that, as was the case for female youths, the JTPA Title II-A program did not have a positive 
impact on the earnings or employment of male youths. 

A Summary of Impacts, in the Context of Previous Research 

All told, the estimated impacts of JTPA Title II-A on out-of-school youths present a picture 
of a program that was not working to increase the earnings and emploN'ment of those out- 
of-school youths at the 16 sites studied but that v^as achieving success in increasing the 
likelihood that those youths without a high school credential would obtain one over the 
18 months of follow-up that we have studied so far. This section summarizes the findings 
for youths in more detail and then compares them with results fi-om the two previous 
experimental studies of employment and training programs for out-of-school youths. The 
findings from the three studies are not inconsistent with one another, and together they form 
the most reliable body of knowledge that is available on the effectiveness of emploNonent 
and training programs for economically disadvantaged out-of-school youths. 



Female Youths 

As summarized in the first two rows of Exhibit 6.16, JTPA had a negligible impact on 
the earnings of the 2,323 female youths in the study sample: It reduced earnings $-182, 
or -2.9 percent over the 1 8-month follow-up period. The loss was not statistically significant, 
however. 

Although the estimated impacts on earnings differed across the three service strategy 
subgroups of female youths (column 2 through 4), again none was statistically significant. 
Specifically, the estimated impacts were a loss of $-542 (-9.1 percent) for the classroom 
training subgroup, a gain of $410 (5.4 percent) for the OJT/JSA .. bgroup. 
and a loss of $-158 (-2.8 percent) for the other services subgroup. 

The next five rows of Exhibit 6.16 show that the estimated impacts on employ went 
were also for the most part negligible for female youths. The estimated impacts on 
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Exhibit 6.16 Summary of Estimated JTPA Jmpaas on Earnings and Employment over 
the Full Follow-up Period: Out-of-School Youth JTPA Assignees in the 
18-Month Study Sample, by Gender and Service Strategy Subgroup 







Classroom 


OJT/ 


Other 




All 


training 


JSA 


services 


Impact per 


subgroups 


subgroup 


subgroup 


subgroup 


assignee on: 




a) 


(3) 


(V 






Female youths 




Earnings 










Iq$ 


$ -182 


$ -542 


% 410 


$ -158 


As % of control mean 


'2.9% 


-9.1% 


5.4% 


-2.8% 


Percentage employed 


2.8% 


3.5% 


3.6% 


2.6% 


Weeks worked 










In weeks 


-1.6 


-2.4 


-1.4 


-0.4 


As % of control mean 


-4.6% 


-7.2% 


-3.4% 


-1.3% 


Hours worked 










In hours 


-61 


-119* 


-13 


-4 


As % of control mean 


-4.7% 


-9.7% 


-0.8% 


-0.4% 


Sample size ^ 


2^00 


L045 


545 


*7 in 
/lU 






Male youths 




Earnings 










In$ 


$ -854** 


% -259 


$-1,313* 


$-1,305* 


As % of control mean 


-7.9% 


-2.6% 


-iO.3% 


-13.3% 


Percentage employed ^ 


1J% 


2.4% 


0.9% 


1.6% 


\\'eeks worked 










In weeks 


-2.3* 


-1.3 


-2.3 


-3.0 


As % of control mean 


-4.9% 


-2.9% 


-4.5% 


-7.1% 


Hours worked 










In hours 


-129** 


-50 


-182 


-169 


As ^ of control mean 


-6.8% 


-2.9% 


-8.3% 


-9.6% 


Sample size ^ 


1,748 


526 


615 


607 


Source: Estimates based on First Follow-up Survey responses. 






Note: Estimates are regression-adjusted to control for differences in baseline characieristics between the 



treatment group and control group; sec Appendix D. 

a. At any time during the 18*month follow-up period. The impaci is measured in percentage points. 

b. Treatment and control groups combined. 

» Sutistically significant at the .10 level, »» at the .05 level, »»• at ihe .01 level (two-tailed test). For the 
impacts on earnings, weeks worked, and hours worked, the signiftcance level of each estimate 
expressed *as % of control mean* is the same as that of the corresponding estimate expressed in dollars, 
in weeks, or in hours. 



ERIC 



228 • JTPA 18-MONTH IMPACTS / OUT-OF-SCHOOL. YOUTHS 



emplovment rates were small but positive for female youths overall, at 2.8 percentage pomts. 
and for all three other services subgroup to 3.6 percentage points for the OJT/JSA subgroup. 
The impacts on the number of weeks and hours worked, averaged over all sample members, 
were however, negative for the target group overall and for all three sen-.ce strateg> 
subgroups. But none of these impacts on employment was statistically significant, except 
for the -119 (-9.7 percent) loss in hours worked per week worked for female youths in 
the classroom training subgroup. Together these findings suggest that JTPA had no 
significant impact on employment for female youths-net in the ability to find a job. how 
quickly jobs were found or how long they were held, or the extent of fiiil-time jobs-but 
the program did have an effect on the extent of full-time work for those female youths 
who were recommended for classroom training." 

Despite the negligible impacts on earnings and employment for female youths, access 
to JTPA had a highly significant impact on attainment of a training-related high school 
diploma or GEO certificate, for female youth overall and for those in the classroom training 
and other services subgroups— the two subgroups in which basic education was a key service 
received. Among those female youth who were high school dropouts upon application to 
JTPA the estimated impacts on this form of educational attainment were 1 1 .9 percentage 
points for female youths overall and 16.4 and 10.7 percentage points for the classroom 
training and other services subgroups, respectively. Recall that the JTPA performance 
standards for vouths are in essence Uvofold, emphasizing both emploNment outcomes and 
education and'training outcomes. Hence, the estimated impact on high school attainment 
relates to a central objective of the program. 

Forthe kev subgroups of female youths, selected fortheir relevance to policy and program 
planning debates, none of the estimated impacts on 18-month earnings was statisticalh- 
significant; nor were any of the subgroups within a related set of subgroups significantly 
different from one another. 

Male Youths 

As shown in the bottom panel of Exhibit 6.16, JTPA had a negative impact on the earnings 
of the 1,773 male youths in the study sample; It reduced their earnings by $-834, or 
-7.9 percent over the 18-month follow-up period. That loss, which represents an average 
over all male youth treatment group members, was statistically significant. 

But most of the negative impact on earnings (82 percent) was concentrated in the 25 
percent of male youths who reported having been arrested between age 16 and their random 

31 Exhibit H 4 in Appendix H, which displays estimated impacts on the components of the earnings 
impact for female youths overall, suggests that the impact on earnings represents a combmation of a 3.6 
percent impact on the employmem rate and a 1.9 percent impact on earnings per hour worked, which 
together are offset by a -7.9 impact on weeks worked per worker. 
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assignment. Members of this subgroup experienced a highly significant loss of $-3,038, 
on average, or 27.0 percent of v*at they would have earned without access to JTPA. In 
contrast, the male youths who reported no previous arrests {75 percent of the total) 
experienced an estimated $-224, or 2. 1 percent, earnings loss. Hence, the impact on earnings 
for most male youths was similar to that for female youths: It was negligible. 

The estimated average impacts on earnings were negative across all three service strategy 
subgroups of male youths, but they were most native for the OJT/JSA and other services 
strategies. Male youths recommended for the OJT/JSA strategy had an earnings loss of 
$-1,313 (-10.3 percent), those recommended for the other services strategy had an 
earnings loss of $-1,305 (-13.3 percent). Both of these estimated impacts were statistically 
significant. Male youths recommended for the classroom training strategy, on the other 
hand, had a statistically insignificant loss of only $-259 (-2.6 percent). In all three service 
strategy subgroups, however, those male youths with a previous arrest experienced more 
extreme negative impacts than did those without a previous arrest.^^ 

The last five rows of Exhibit 6.16 present mixed evidence of JTPA impacts on 
employment for male youths. The estimated impacts on employment rates were small, 
positive, and not statistically significant for male youths overall and for all three service 
strategy subgroups. The impacts on the number of weeks and hours worked averaged 
over all sample members, were, however, negative and statistically significant. 

Male youths overall appear to have experienced a loss of -2.3 weeks of work (-4.9 
percent) and a loss of -129 hours of work (-6.8 percent). The estimated impact on the 
average number of weeks and hours woiked were also negative for all three service strategy 
subgroups, but they were not statistically significant. Together these findings, alcmg with 
those on program impacts on the components of earnings in Exhibit H.10 (Appendix H), 
suggest that the earnings loss for male youths was primarily a result of negative impacts 
on the ability to fmd a job quickly, how long jobs were held, and the extent of fiill-time 
employment, not the result of a negative impact on the ability to find a Job per se. 

Despite the negligible impacts on earnings for most male youths and the extremely 
negative impacts on earnings for those with a previous arrest, JTPA again had a highly 
significant impact on attainment of a training-related high school credential for male 



32. As noted earlier in the chapter* the findings for male youths with a previous arrest that are based 
on First Follow-up Survey responses are inconsistert with findings for a subsample of male youths estimated 
based on earnings daU torn sUte unemployment insurance agencies (see Appendix £). Our foithcoming 
final report will further investigate this discrepancy, but note that neither data source yields fmdings 
suggesting positive impacts on earnings for male youths with or without a previous arrest. Also note 
that the impact fmdi.igs from the two data sources were not inconsistent for adult women, adult men, 
female youths, and male youths who did not have a previous airest. 
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youths, as it did for female youths. The impacts were positive and statistically significant 
for all three seivicc strategy subgroups. Specifically, for male youths who were high school 
dropouts the likelihood of attaining a training-related high school credential w^ increased 
by 9.9 percentage points for male youths overall and by between 9.0 and 10.1 percentage 
points for each of the three service strategy subgroups. 

Finally, we have already mentioned the most prominent finding firom our analysis of 
key subgroups of male youths, namely, that most of the negative average impact estimated 
for male youths overall was concentrated among those \^^io reported having been arrested 
since turning the age of 16. 



Previous Experimental Findings on Programs for 
OuT-oF-ScHooL Youths 

To provide a broader context in which to assess the impacts reported above. Exhibit 6.17 
displays our 18-month findings for JTPA along with findings fi-om the two other major 
randomized experiments that have been conducted to date on employment and training 
programs for out-of-school youths: the JOBSTART demonstration (Cave and Doolittle, 
1991), and the youth component of the National Supported Work Demonstration (Maynard, 
1980). 

Columns 1 and 2 show that although the three studies have much in common, the samples 
of out-of-school youths studied differed somewhat across the three studies, and services 
provided to those youths differed even more. JOBSTART services, for example, were similar 
to those in JTPA but were more intensive and did not include on-the-job training, whereas 
Supported Work offered structured, paid work experience exclusively, a service that was 
rarely provided to the out-of-school youths in JTPA. The comparison below is therefore 
only in the most general of ternis. 

Columns 3 through 5 present the impact estimates. As in exhibits 4.17 and 5.16 in 
chapters 4 and 5, the estimates are expressed as average quarterly impacts during each 
of the three years following sample members' random assignment and the estimates fi'om 
the previous studies have been converted to July 1989 dollars. 

The JOBSTART demonstration, conducted at 13 sites in the mid-1980s, tested an 
intensive program ofbasic education, classroom occupational skills training, support services 
(mainly child care and transportation), and job search assistance for economically disad- 
vantaged high school dropouts, ages 17 to 21, who read below the eighth grade level. This 
study sample was about evenly divided by gender; and 5 percent of the females in the treatment 
group and 29 percent of the males had a previous arrest. 
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Exhibit 6.17 Esnmaied Impacts on the Average Quarterly Earnings of Out-of-SchooI Youths: 

The JTPA JS'Month Impact Analysis and Previous Experimental Studies 
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The JOBSTART findings for the first two years after random assignment basically 
parallel our JTPA Tjidings. The estimated impacts on the earnings of female youths were 
negligible and not statisticaUy significant in years 1 and 2 ($66 and $58), whereas the impacts 
on the earnings of male youths were negative in both years ($-284 and $-177) and statistically 
significant in the second. 

These estimates are of particular interest not only for their similarities to the estimates 
for JTPA shown in the top panel, but also because of certain similarities between JOBSTART 
and JTPA participants and programs. About half of the outK)f-school youths in the JTPA 
18-month study sample were high school dropouts, and 5 percent of the females and 25 
percent of the males in the treatment group were previous arrestees. Moreover, the 
JOBSTART demonstration was conducted within the JTPA system, and its basic services— 
though more intensive than the average for JTPA— were similar in many ways to those 
in its program for youths. 

The National Supported Work Demonstration, a component of which was conduaed 
with outK)f-school youths in five sites during the late 1970s, studied a lengthy, intensive, 
andhighlystr ctured program ofpaid work experience. The study treatment group comprised 
economicall> disadvantaged high school dropouts, ages 17 to 20, who were mostly male 
(88 percent) and a large proportion of whom (57 percent) had a previous arrest. 

The exhibit does not show impaa estimates for the first year after random assignment 
for Supported Work because most treatment group members were in the program and 
receiving heavily subsidized earnings during that period. The estimates for the second and 
third years after random assignment, when most treatment group members were out of the 
program, were negUgible ($22 and $-42) and statistically insignificant. 



Source* for Exhibil 6.17: , ^ ^^^^ 
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Thus, despite important differences in the program participants and services studied 
in the three analyses, their comparison is nonetheless instructive. The three programs studied 
represent a broad range of efforts to inq}rovc the earnings and employment for out-of-school 
youths, and all three studies used a rigorous evaluation methodology. Taken together, these 
studies lead to the conclusion that our nation has not yet found a solution to the employment 
and training problems of its economically disadvantaged out-of<>school youths. 



7 



A Comparison of JTPA Impacts across 
Target Groups and Study Sites 



THE three previous chapters examined the estimated effects of JTPA on four different 
target groups — adult women, adult men, female youths, and male youths — taking 
each target group in turn. This chapter reviews the main findings of those chapters side 
by side, looking for similarities and differences. Of particular interest is the sharp contrast 
between the estimated impacts on adults and youths. The chapter also presents separate 
findings on program impacts across the 16 study sites, which allow us to consider whether 
the local JTPA programs differed in impacts and to identify local program characteristics 
that led to the largest impacts. 

We do not present estimates of the effects of JTPA on all target groups combined 
precisely because of the important differences in the program impacts bervveen adults and 
youths and, to a lesser extent, between females and males. Indeed, the differences are 
so fundamental as to render estimates of average program impact on the combined sample 
essentially meaningless. The compare-and-contrast approach followed here therefore seems 
the best way to tell the stor>' of the program's overall effectiveness at the study sites. 

The chapter begins by examining the effects of JTPA on earnings and emploNTnent 
in each of the four target groups. It then proceeds to compare the experiences of the 
three service strategy subgroups in each target group — the classroom training, OJT/JSA, 
and other services subgroups — examining for each the employment and training services 
received and the impact of those services on earnings and the two main components of 
earnings (hours worked per assignee and earnings per hour worked). Later sections of 
the chapter examine impacts on key subgroups (such as those facing the employment 
barrier of welfare receipt or limited education) within each target group and variations 
in impacts across sites. 

235 
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The comparisons presented here are both statistical and descriptive in nature, making 
the analysis as a whole somewhat more tentative tlian that of the three preceding chapfers. 
Only the most important comparisons across target groups and study sites are tested for 
statistical significance, in part because comparisons between target groups or sites ha\ e 
less power to detect real differences than tests of each target group individualK .' Hence, 
one should not be surprised that only a few of these comparisons yielded statistically 
significant results. 



Impacts on Earnings and Employment Overall 

Exhibit 7.1 shows the estimated effects of JTPA on earnings, the percentage employed, 
and the average number of weeks and hours vorked per assignee for each of the four 
target groups over the fiiU 18-month follow-up period. For each target group the exhibit 
displays both the mean outcome for the control group and the impact per JTPA assignee. 
The impact per assignee (treatment group member) is defined as the difference between 
the treatment group mean and the control group mean. Recall from earlier chapters that 
these estimates reflect the average effects of access to JTPA on all assignees, or more 
specificall}', how much higher (or lower) earnings and emploNinent levels were for those 
who had access to the program than for those who did not. 

Also note that all of the impact estimates in the exhibit refer to JTPA assignees overall— 
whether they became enrolled in JTPA or not. As explained in eariier chapters, the inferred 
impact estimates for the narrower population of assignees who did become enrolled were 
uniformly larger than those shown here for assignees, if one assumes that the nonenrollees 
experienced no program effects. For simplicity we confine our attention in this chapter 
to impacts per assignee. The inferred impacts per enrollee (under the assumption of 
no effects on nonenrollees) that appeared in chapters 4 through 6 exceeded the impacts 
per assignee by 59 percent to 7 1 percent, depending on the target group. Thus, comparisons 
across target groups would change only slightl>- if we used the per enrollee estimates 
instead of the per assignee estimates. 



1. Tests of one target group against another arc subject to greater sampling variability Uian tests 
of a single target group, since they are based on two sample-based estimates in.stead of one. The same 
is true of tests comparing different sites which are also hmited by the small samples available from each 
site. 
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Impacts on Earnings: JTPA Assignees Overall 

Because the four target groups differed substantially in their baseline characteristics (see 
e>diibits 3.13 and 3.14 in Chapter 3), we would expect control group earnings over the 
18-month follow-up period to van- considerably by target group. As shown in the first 
row of Exhibit 7 . 1 , the male control groups earned much more than their female counterparts, 
and the adult control groups earned somewhat more than their youth counterparts. It 
is these findings for the control groups that represent our estimates of what JTPA assignees 
would have earned without access to the program. 

The four target groups also differed somewhat in the JTPA services they were rec- 
ommended for and received, as will be shown later in the chapter. Together the differences 
in individual characteristics and services recommended and received led to substantial 
differences in the estimated program impacts on earnings for the four target groups. At 
one extreme is the positive and statistically significant impact of $539 on adult women's 
earnings (7 percent of the control group mean). The estimated impact on adult men was 
similar, at $550, although that figure was not statistically significant at conventional levels 
and represented a smaller percentage (4.5 percent) of the control group mean.- At the 
other extreme is the negative and statistically significant impact of $-854 on male youths' 
earnings (or -8 percent of the control group mean) — a finding that stems mainly from 
negative impacts on male youths with a previous arrest and that is statistically significanth' 
different from the positive estimated impacts on both of the adult target groups.^ Female 
youths seemed to experience little or no program effect on earnings, a fmding that again 
is significantly different from the results for both of the adult target groups. Impacts 
did not differ significantly by gender for either adults or youths. 



2. As discussed in Chapter 5, despite this lack of statistical significance, the estimated impact on 
adult men is likely to reflect a true impact, for several reasons. First, the estimated impacts on adulf 
men's earnings were positive in all six follow-up quarters and for all three service strategy subgroups 
Second, the $550 impact estimate for the full 18-monlh pcnod was statistically significant at a near- 
conventional level (.15). Finally, and most tellingly, the estimated impacts on all three measures of employment 
(the percentage employed and the number of weeks and hours worked) for adult men were positive and 
statistically significant. 

3. Recall from Chapter 6 that the extremely negative survey-based estimate of the impact on the 
eammgs of male youths with a previous arrest — and thus the very negative estimated impact on earnings 
for male youths overall — is not supported by impact estimates based on an alternative data source, namely, 
earnings data from state unemployment insurance agencies. The survey-based estimates presented here 
should therefore be viewed with caution until further analyses appear in our forthcoming fmal report 
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Impacts on Employment: JTPA Assignees Overall 

The remaining rows of Exhibit 7,1 display the estimated effects of JTPA on three different 
employment outcomes: the percentage of assignees employed at any time during the 18- 
month follow-up period and the average number of weeks and hours worked during that 
period/ Again there are substantial differences in impacts between the adults and the 
youths. 

Adult women in JTPA experienced a significant increase in overall employment, up 
2.1 percentage points from the control group mean of 76,8 percent. Estimated increases 
in weeks and hours worked were of a similar magnitude (not shown, but around 3 percent 
of the corresponding group control mean) but not statistically significant. For adult men 
the three estimates of impacts on employment were slightly higher, and all three were 
statistically significant. JTPA increased overall adult male employment by an estimated 
2.8 percentage points and resulted in 2.2 more weeks and 84 more hours of employment 
over the follow-up period. Each of these effects represents a 3 percent to 5 percent increase 
over the corresponding control group mean (not shown)— again, similar to or slightly larger 
than the estimated effects for adult women. 

On the other hand, no statistically significant increases in employment are evident 
among the youths, although the estimated impact on the percentage employed was 
insignificantly positive for both females and males. The estimated effects on weeks and 
hours worked were negative for both youth target groups, with those for the males 
statistically significant and of a greater magnitude than those for the females. Specifically, 
these estimates show a drop in work time of 2.3 weeks and 129 hours for male youths, 
or 5 percent to 7 percent of the corresponding control group mean (not shown). For both 
genders the estimated effects on weeks and hours worked were significantly more negative 
for youths than for adults. Estimated effects on the three employment measures did not 
differ significantly among target groups in any other instance. 

In summary, then, these findings of impacts on employment provide a fuller picture 
of the differences in program impacts across target groups than simply the earlier evidence 
on earnings. The gains among adults have been clarified in two respects. For adult women 
we find that the gains in earnings were accompanied by an increase in the proportion 
of women who worked at some time during the 18-month follow-up period. For adult 
men Lhe evidence of program benefits is much stronger once we examine the three 
employment outcomes, all of which were significantly increased by access to JTPA. 



4, Both the weeks worked and the hours worked estimates in Exhibit 7. 1 are averages for the group 
in question; that is, the estimates include zero values for those sample members whc did not wrk at all. 
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Among youths we see that JTPA did not reduce the percentage of the males ever 
employed during the 18-month follow-up period, as it did the other labor market outcomes 
considered. No significant employment effects occurred for female youths. 



Services Recommended and Received 

These differences in labor market impacts across the target groups could have arisen from 
either or both of two sources: differences across target groups in JTPA services 
recommended and received, or differences in the ability of the various target groups to 
benefit from a given set of services. For example, the sendees adults and youths received 
may have differed because of differences in the JTPA performance standards which 
emphasize employment for adults but also include educational attainment for youths. This 
study was not designed to trace differences in impacts to differences m programs versus 
differences in participants, but to measure the net effect of the two taken together as 
they naturally occur within the JTPA system. We can, however, provide some insight 
into the influence of service differentials by comparing subsets of individuals from each 
target group who were recommended for the same service strategy. 



Service Strategies Recommended: Treatment Group 

Exhibit 7 2 shows that the four target groups did, in fact, differ substantially in the mix 
of service strategies for which they were recommended.^ Among both adult women and 
female youths classroom training was the most frequently recommended semce strategy; 
among men it was the OJT/JSA strategy; and among male youths it was the other services 
strategy. For the classroom training strategy, SDA staff recommended 44 percent of 
all adult women and female youths, but only a quarter to under a third of male youths. 
The staff were much more apt to recommend adults— both women and men— for the 
OJT/JSA strategy, whereas the reverse was true of the other services strategy. A particularly 
noticeable difference was that adult men were much more likely than all three of the 
other target groups to be recommended for on-the-job training (the OJT/JSA strategy). 



5. Statistical tests of these differences were not conducted, however. 
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Exhibit 7.2 Service Strategies Recommended: Treatment Group, by 
Target Group 



Recommended 


AduU 


AduU 


Female 


Male 


service 


"women 


men 


youths" 


youths" 


strategy 


(1) 


a) 


(3) 


(4) 


Classroom training 


44.0% 


24.6% 


44.3% 


29.9% 


OJT/JSA 


35.0 


48.7 


23.2 


32.9 


Other services 


21.0 


26.7 


32.5 


37.3 


Sample size 


4,465 


3.759 


1.814 


1,436 



Source: Unadjusted frequencies based on Background Information Form responses. Sec 
Exhibit 3.16. 

a. Out-of-school youths only. 



JTPA Services Received: Treatment Group, 
BY Service Strategy Subgroup 

As shown in the three preceding chapters, the service strategy recommendations of program 
staff resulted, in turn, in subgroups of JTPA enrollees who received distinctly different 
clusters of specific program services. Moreover, because members of each service strategy 
subgroup could receive more than one service, a given subgroup could differ substantially 
across Urget groups in terms of the mix of specific program services received. Thus, 
a comparison of differences in the mix of services received across target groups within 
each service strategy subgroup will lend insight into our later presentation of program 
impacts by se-vice strategy subgroup. 

Exhibit 7.3 shows the percentage of treatment group members who received various 
JTPA services during the 18-month follow-up period, by target group and service strategy 
subgroup.^ Differences within each service strategy subgroup were modest but potentially 
important.' 



6. Percentages sum to more than 100 percent within columns because sample members could receive 
more that one employment and training service. As explained in earlier chapters, most Ucalmcnt group 
members who failed to receive one o\ key services in their service strategy subgroup did so because 
they were never enrolled in JTPA. Thus, service receipt rates among JTPA enrollees were substantially 
higher than those shown in Exhibit 7.3. For the most conunon service received in each service strategy 
subgroup these rates of receipt per service receipt ranged from 70 percent to 79 percent in the classroom 
training subgroup (for classroom training in occupational skills), from 50 percent to 52 percent in the 
OJT/JSA subgroup (for on-the-job training), and from 48 percent to 52 percent in the other services subgroup 
(for miscellaneous services). 

7. These differences were f^ot tested for statistical significance. 
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Exhibit 73 Receipt of Specific JTFA Services: Treatment Group, by Target 
Group and Service Strategy Subgroup 



Specific program 
service 



Never enrolled 
Classroom training 
in occupationid skills 
Basic education^ 
On-the-job training 
Job search assistance 
Woric experience 
Miscellaneous ^ 



' AduU 
women 
(I) 



%of treatmen t group receiving the sendee 
7[3nU Female Male 



men 

(2) 



youths 
(3) 



211% 



Classroom training subgroup 
28.8% 285% 



youths ^ 
(4) 



57.8 


55.7 


54.8 


52.4 


10.6 


8.8 


17.8 


233 


3.3 


5.4 


2.6 


4.4 


17.1 


12.4 


2', 3 


30.8 


3.9 


1.7 


5.7 


6.5 


113 


9.7 


7.7 


7.9 




OJTUSA subgroup 




44.6% 


43.4% 


423% 


413% 


5.1 


2.1 


3.3 


1.9 


2.6 


3.6 


3.1 


2.8 


285 


26.6 


29.9 


303 


265 


302 


283 


322 


2.6 


2.4 


52 


4.2 


5.8 


6.8 


7.1 


6.8 




Other services subgroup 




37.6% 


41.1% 


36.9% 


323% 



Never enrolled 
Classroom training 

in occupational skills 
Basic education^ 
On-the-job training 
Job search assistance 
Work experience 
Miscellaneous^ 

Never enrolled 
Classroom training 
in occupational skills 
Basic education ^ 
On-the-job training 
Job search assistance 
Woric experience 
Miscellaneous ^ 
Sample size 



15.6 


4.9 


9.8 


6.5 


11.1 


6.1 


29.7 


26.4 


5.5 


4.7 


3.9 


3.9 


23.4 


24.8 


122 


12.0 


2.7 


0.9 


3.4 


3.2 


315 


28.4 


283 


353 


4.465 


3,759 


1.814 


1.436 



Souree: EmoIImenl .nd U.cking <Uu fn.m the 16 .e.vice deliveor (SDA.). See Exhibit 3.18. 
a. Oul-of-ichool yoolh* only. ^ , , 

b •B..ic eduction- include. Adult B.«c Eduction (ABE). hi,h .chool or Gene«I Educt.on.1 

Development (GED)pf«pirition.indEngli.h It 1 Second L«itu«ge(ESL). . , , .. 

^MiKen.neou.- i Jude. ..ment. jo»>«Klineu training, customized tr..rang. voc.t.on.I explor...on. 
job thidowing, ind tiyoul employment, imong other lervicet. 
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Within the classroom training subgroup around 55 percent of the treatment group 
in all four target groups received the defining service for the service strategy— classroom 
training in occupational skills. Treatment group members m the youth target groups were, 
however, much more likely to receive basic education and job search assistance than their 
counterparts in the adult target groups— with 20 percent of youths versus around 10 percem 
of adults enrolled in basic education courses and around 30 percent of youths versus 
around 15 percent of adults enrolled in job search assistance. 

No important differences in service receipt across target groups occurred for those 
in the OJT/JSA subgroup. For all four target groups enrollment in each of the two key 
services in the strategy— on-tl»e-job uaining and job search assistance— was around 30 
percent. 

The largest contrasts occurred within the other services subgroup, which, for adults, 
was characterized by receipt of miscellaneous services and job search assistance and, 
for youths, by receipt of misceiianeous services and basic education. Specifically, the 
adults (at around 24 percent) were more likely than the youths (at 12 percent) to receive 
job search assistance (which is the opposite of the results for job search assistance m 
die classra^m training subgroup). The youths, on the other hand, were much more likely 
(at arounci 28 percent) than the adults (between 6 and 1 1 percent) to receive basic education. 
Even the service category that had the highest enrollment rates in all four target groups 
showed some variation. About 28 percent of adult men and female youths received 
miscellaneous services, whereas the corresponding figures for adult women and male youths 
were 32 percent and 35 percent. And adult women in the other services subgroup were 
much more apt to become enrolled in classroom trainmg m occupational skills than any 
other target group (16 percent versus a range of 5 to 10 percent). 

Finally, within each service strategy subgroup differences in overall enrollment in JTPA 
and in the average duration of enrollments were slight. As shown in the first row of 
each panel in Exhibit 7.3, the percentages of the treatment group who were never enrolled 
in JTPA over the 18-month follow-up period were very similar across target groups in 
both the classroom trainmg subgroup and the OJT/JSA subgroup. In the other services 
subgroup the female target groups had virtually the same rate of nonenrollment (about 
37 percent), but adult men were more likely (41 percent) and male youths less likely 
(32 percent) to never enroll. And as shown earlier (Exhibit 3.20 in Chapter 3), the average 
durations of program enrollments within each service strategy subgroup were also quite 
similar across target groups, with the minor exceptions of slightly shorter average enrollment 
rates among the males than the females in the classroom training and other services 
subgroups.* 



8, These differences were not tested for statistical significance. 
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All told then, the only substantial differences in services received and enrollment vsnthin 
each service'strategysubgroupagaindistinguishthcyouthsfromthc adults. In A^^^^^^ 

training subgroup the youths were much more likely to receive basic education and job 
search assisLce. and in the other services subgroup the youths were apt to receive more 
basic education but less job search assistance than the adults. 

Services Received from Any Provider: Treatment akd Control Groups, 
BY Service Strategy Subgroup 

Although informative, the above comparisons of JTPA services recoromended and received 
across target groups do not present a complete picture of the differences m semce receipt 
S^It created L labor market impacts shown earlier. Bear in mind that those impacts 
were produced by differences in service receipt between the treatment group and control 
group members within each target group-service stratc^ subgroup combination-dif- 
ferences that go beyond those measured by the JTPA services received by treatment group 
members alone. Control group members also received employment and traming services 
during the follow-up period, from non-JTPA sources, as did some members of the Ueatment 
group. 

Exhibit 7 4 summarizes what we know about treatment-corJrol group differences 
in service receipt from any provider, by target group within each service stiategy subgroup. 
The exhibit focuses on the average amount of each service received (in hours, including 
zero hours for those who received no services) as the best overall measure of service 
receipt » As explained in Chapter 4, we were unable to obtain reliable data on the receipt 
of job search assistance and miscellaneous services from non-JTPA service providers. 
Nor were we able to obtain data on the receipt of on-the-job training and work experience 
from non-JTPA providers, although in these insunces we are fairly confident the JTPA- 
only dau shown in the exhibit represent nearly all of the hours of serWce received since 
on-the-job training and work experience are seldom available from other providers. 

The differences in the average number of hours of service receipt shown in Exhibit 
7 4 are diluted by the high proportion of both treatment and control group members who 
received zero hours of each type of service. Hence, they appear small even though they 
may represent substantial additional amounts of services for service recipients within the 
treatment and control groups. 



9 Additional details on the likelihood (percentage) of the ueatment and control groups receiving 
each sendee Td on the number of hours of each service received by the treatment and control groups 
separately, appeared earlier in exhibits 4.8, 5.7. 6.4 and 6.9. 
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Exhibit 7.4 Difference in the Mean Number of Hours of Employment and Training 
Services Received by the Treatment Group and Control Oroup: 
Target Groups, by Service Strategy Subgroup 



Specific program 
service 



Classroom training in 

occupational skills ^ 
Basic education ^ * 
On-the-job training'** 
Work experience'** 
Job search assistance 
MisceUaneous'*** 
Sample size ^ 



Classroom training in 

occupational skills * * 
Basic education^ * 
On-the-job training* * 
Work experience '** 
Job search assistance * "* 
MisceUaneous '*** 
Sample size / 



Treatment-control group difference 



Adult 
women 

(1) 



AduU 
men 

a) 



Female 
youths^ 
(S) 



Classroom training subgroup 



2,847 



1,057 



1,045 



-2 
7 
104 
13 



2,287 



OJT/JSA subgroup 

3 -16 

-2 -2 

114 105 

0 17 



2,250 545 



Male 
youths ^ 

(4) 



110 


95 


187 


127 


9 


9 


36 


16 


25 


37 


24 


18 


23 


8 


32 


25 



526 



58 
-32 
128 

10 



615 



(Coniinued) 



Even with this dilution, youth treatment group members in the classroom training 
5w6graw/>— especially female youths— held a noticeable advantage over their control group 
counterparts in hours of classroom training in occupational skills: differences of 187 
hours for females and 127 hours for males. The only clear treatment-control group difference 
in hours of b?,sic education received— 36 hours— also occurred within the female youth 
target group. Both of these differences for female youths represent a doubling of the 
average amount of the services received by the corresponding members of the control 
group, though not a large gain in terms of the total number of hours of service receipt. 
No other target group received so large an increment in services when recommended for 
classroom training, in either hourly or percentage terms." 

10 Among service recipients only (roughly two-thirds of the female youth treatment group), the 
diflerence in the average amount of classroom instruction of either type that was received was substantially 
larger, though still modest: 330 total hours of classroom training in occupational skills and basic education, 
rather' t\an the combined total of 223 hours shown in Exhibit 7.4. 

11. No tests were performed to determine if the increment in services received by female youth 
treaim'int ^roup members in the classroom training subgroup significantly exceeded that of the classroom 
training subgroup in the other three target groups. 
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Exhibit Z4 Difference in the Mean Number of Hours of Employment end Training 
:Ser^ices Received by the Treatment Group and Control Group: 
Target Groins, by Service Strategy Subgroup (continued) 

Treasmeni'Control group difference 

'^idh^ Female Male 

^ youths" youths'^ 

a) (S) {fl— 

Other services subgroup 



AduU 



Specific program women 

(1) 



service 



Classroom training in 
occupational skills 
Basic education ' * 
On-the-job tiaining'* * 
Work experience'** 
Job search assistance* 



Sample size ^ 



18 


A 


I 


10 


7 


8 


3 


-I 


35 


27 


21 


14 


18 


6 


3 


8 










L340 


L112 


710 


607 



urbmSkd.uoo««p»ofthU«mce. Atthouth U.e d«. «r for rfPA T.Uc U-A-Junded 
KTvice* only, Out «»vice i» typicilly not fimded by ooo-nTA ptwiden. 

+ + + JT^ti «c «ported in IbU becu« d.u were «o» .vtiWIc oo Hce.pt of Ou. «v.ce froo, 
oihcr Drovidera or oo receipt by control group meoiber*. 

Not«: d-u, «^7e «c for .cvicc c^uUua f«« diffcroU d.u tourcc may v*o^ . 

Tcju of «*li»l»cAl »itnific«fice we»r oot performed for thU exhibit. 

fl. Oui-of-tcbool youihtooly. 

.B^ee.u«U<.-^.-ac.Aau>.B«.^»(ABE).^^.boo,or 
Ge««i Edu«uoo;J De*elopoa.t (OED) p«p«.u<«. M<J Eotbd. u . S«:ood Unp-je (ESL). 

.< Houn «Mumine«fall-Uiiiejot)»l40hour«perweek. 

1" S!i^eou?ioc:.ude* J«n«it, jo^f«di«« imnin,. cu.tc«ixea imnmc. voc*lK».l explor.uoc. 

job »b«dowiin<. and tiyout cmploymait. amoot other »cfvice». 
/. TftJtmcnl ind control group* combined. 



Treatment-control group differences in service receipt are not so easily assessed for 
the key services in the other two service strategy subgroups. Data on service receipt 
from non-JTPA providers are unavailable for job search assistance (a key service in the 
OJT/JSA subgroup and, for adults, in the other services subgroup) and for miscellaneous 
services (a key service in the other services subgroup). For the OJT/JSA subgroup we 
can at least compare the treatment group's advantage in the amount of on-the-job traintng 
received across target groups, on the assumption that OJT is available only through JTPA. 
•niat comparison reveals an almost uniform treatment group advantage ranging from 104 
to 128 hours across target groups.'^ This lack of variation across Urget groups is consistent 

12 f^o^o service recipients onTy the difference in the average amount of OJT received was nearly 

four LfTs g?e^ diff"-nceo shown here combine a difference of around 400 hours per serv.ce 

edp eTfor tfe 27 percent to 30 percent of OJT/JSA treatment group members who actually became 
Sled in on with a difference of 0 hours for the remaining 70 percent to 73 percent. 
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with the ea.rlier finding that the OJT/JSA service strategy subgroup vaned least across 
target groups in terms of JTPA services received. The variation in treatment-control group 
differences in the receipt of basic education-a key service for youths in the other services 
subgroup— W3S also slight between female and male youths. 

Thus, in general, the increments in the amount of service receipt attributable to access 
to JTPA were modest and quite similar across the target groups, although noticeable 
differences did arise in the classroom training subgroup. 

High School Attainment: Treatment and Control Groups 

As part of the school or training services listed in Exhibit 7.4, a number of sample members 
gained a high school diploma or GED certificate. Exhibit 7.5 shows the percentages of 
the treatment and control group members in each target group-service strategy 5ubgroup 
combination who both participated n school or training and achieved a high school credential 
during the follow-up period. We refer to this outcome as the attainment of a training- 
related high school credential. For simplicity we confine our attention here to attainment 
rates for the treatment group overall rather than considering the rates for both treatment 
group members and the high school dropout subgroup presented in the three earlier chapters. 

Not surprisingly given their advantage in receipt of basic education services, youths 
recommended for the classroom training and other services strategies were substantially 
more likely to obtain a training-related high school credential than adults recommended 
for those strategies. Fifteen to 19 percent of youth treatment group members in the first 
and third service strategies received a training-related high school credential, whereas only 
3 to 7 percent of their adult counterparts did. These findings refiect in part the substantially 
larger proportion of adults than youths who already had a high school credential when 
they applied to JTPA (see exhibits 3.13 and 3.14 in Chapter 3). 

Treatment-control group differences in attainment for the classroom training and other 
services subgroups were also larger for youths than adu Its. but were positive and statistically 
significant in all four target groups (not shown).'^ No importont differences by gender 
appear in either the classroom training or other services subgroup. 

Adults and youths recommended for the OJT/JSA strategy differed little in attainment 
of a training-related high school credential, even though a substantially greater share of 



13 For adult men within the other services subgroup the treaimcnt-control group difference in high 
school attainment was positive but not statistically significanl. Comparisons o.^ treatment-control group 
differences in attainment between pairs of target groups were not tested for statistical significance 
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youths than adults could have gained a high school diploma or GED certificate during 
the follow-up period (that is, a higher percentage of youths lacked a high school credential 
at baseline). Attainment rates were small (2 to 6 percent) for all target groups in the 
OJT/JSA subgroup.*^ 

Impacts on Earnings across Service Strategy Subgroups 

With these service and educational differentials in mind, we turn now to a comparison 
of JTPA impacts across target groups in each of the service strategy subgroups. This 
comparison offers further insight into differential program elBfects on the target groups 
by focusing on sample members recommended for a similar mix of services. This focus, 
in turn, provides a better understanding of how differences in the personal characteristics 
of the four target groups led to different impacts on earnings. Remember, though, that 
even within a given service strategy subgroup (such as classroom training or other services) 
patterns of service receipt did vary somewhat by target group. 



Impacts on 18-Month Earnings 

We begin by focusing on variations among the target groups in 18-month impacts on 
earnings for assignees in the same service strategy subgroup. The *'all quarters" rows 
of Exhibit 7.6 display these findings. Reflectbg the general differences in baseline char- 
acteristics among the target groups as a whole, control group earnings over the full follow- 
up period differed across target groui>-service strategy subgroup combinations. Males 
earned significantly more than females, and in most cases adults earned significantly more 
than youths. The male-female contrast is particularly striking, with male earnings ranging 
from $9,783 to $12,765 and female earnings ranging from $5,726 to $8,607 in all three 
service strategy subgroups. 

Program impacts also vary in a highly regular feshion across target group-service 
strategy subgroup combinations, with the impacts for adults more positive than those for 
youths in all cases. This contrast is often pronounced in its dollar magnitude, although 
usually not statistically significant.*^ 

14. Cross-target group comparisoiis of high school attainment rates and impacts for high school dropouts 
only show greater variation (see exhibits 4.9, 5.8, 6.5, and 6.10) than the findings for the treatment group 
as a whole reported in this subsection. We report the attainment findings here for the fuh treatment 
group (including those who had a high school credential when they applied to JTPA) to focus on attainment 
of a high school credential as an intermediate step toward achieving the ultimate goal of JTPA: higher 
earnings for all program participants, in this case treatment group members overall. 

15. Large dificrcnces between target groups are less likely to yield statistically significant differences 
m estimates for the individual service su-atcgy subgroups because of the much smaller samples sizes involved 
(approximately one-third the sample sizes used in the original comparison of ihe full target groups) 



31 J 



den 



5 

r 

V 

£? 

I 

■s 

I 

a. 

S 

a 

I 



I 



9 


^1 












11® 



3^ ^ 



II 



V 
w 

o 



i 



2 P 



NO CM 



o * 



vO *n f*> 0\ 
<S ^ 00 On 

fn vo 00 00 



• * 

o a\ O o 

^ 00 in c<i r* op 
cvi ^ 



o©t^O\OONO*^ vN 

00 



i ^ \o fn o ^ 00 hv 



O ''T 

^ ^ 00 00 00 
t** 00 «-< 



o 

00 



^ ^ ^ CM CM C4 ^ 



• cvirjcM-«f^fnNC 

4^ 



Op l^ g >o yn 



m op Ox , ^ 

^ O «n 

^ ^ ci CM CM cm' 
4^ 



o Ov o >f 

^ CM ^ ^ u-i 



I 

•I 

i 

5 



S"^ CM <2 00 CM O 
fi. Irt O ® >0 CM 
O O CM m cn NO 



^ en O 

On O 00 ^ 

^ CM <s 



^ m <7\ On vo 

vn cn 0\ 00 vo vn 

o — ^ 

^ cm" cm" CM CM CM CM 



OW^<MO^» OON. 

in r* 00 ov 
I •-^ 00 



^ 00 ^0 On _ 

^ >3 00 



o 

v» ^ - . f*^ <^ 

On O <M <M 



— CM m W-i NO 
u 



■c " 

< «2 



5 



• • • • 

—1 On On CM 00 CM 

00 r»J O ^ 00 

•1 •I ^ ^ 



m On On p vp o 

CM ^ o 
— • en ^ vTi NO 



— I <M *n ^ NO 



w-i^00VO^CMvr>t>s 
00<M^t^^OiOo> 



4^ 



CM — 
NO o vn 
m ^ NO 



SON On 
On m 

r*^ 00 00^ 
oT 



NO <M 



»r> 00 o 
^ w-i Nn 



en — NO 
On O CM 

o o 



cn r- 

CM 

o o 



On ^ (S 
NO 
» CM ^ 



r* On 
r- CM On 
NO On — 

CM CM 



s;?2 

CM <M 
CM cm" CM 



On CM O CM 
m cn CM CM 

— CM 



Soo 00 — 
On 

^ ^ <M <^ 



V-> 00 Q 
5 

vn VT) On^ 
^ r*" 



— i CM m NO 



it 

II 

s ^ 
^ L 

O « 



•2 

•c 

fl 



11 

ii 



II 

111 

•III 

O H 5: 



ERIC 



JTPA 18-MONTH IMPACTS / TARGET GROUPS AND STUDY SITES ♦ 251 



For the classroom training subgroup, estimated impacts on 18-month earnings were 
positive and statistically insignificant for adults, and negative and insignificant for youths. 
Differences between target groups were not statistically significant, except in the case 
of the two extremes: the $418 positive estimate for adult women and the $-542 negative 
estimate for female youths. 

The contrasts are much sharper in the OJT/JSA subgroup, where most of the statistically 
significant impacts on 18-month earnings by service strategy occurred. Here we find 
estimated earnings gains of $742 for adult women and $78 1 for adult men and an estimated 
$-1,313 loss for male youths (which reflects the extremely negative effects on male youths 
with a previous arrest record as measured by the survey data; see Chapter 6).** The 
estimate for female youths was positive, at $410, but not statistically significant. Despite 
this wide range, only the estimate for male youths differed significantly fi-om those for 
the other target groups. 

The other services subgroup of male youths experienced a similar significant earnings 
loss of $-1,305 (again largely due to negative effects on male youths with a previous 
arrest). Members of the other services subgroup in the other three target groups did 
not experience a significant impact on earnings. The only statistically significant difference 
in effects among these subgroups was between the estimated gain of $457 for adult women 
and the estimated loss of $-1,305 for male youths. 



Impacts on Quarterly Earnwgs 

In addition to these effects on 18-montli earnings. Exhibit 7.6 provides information on 
the time path of earnings and earnings effects within each service strategy and target 
group. We do not present this breakdown into three-month periods, or quarters, for the 
target groups as a whole, since the fi.ndings in chapters 4, 5, and 6 clearly indicate that 
the time paths of the impacts differed substantially across service strategies. Wc consider 
instead whether those time paths also differed by target group within each service strateg>' 
subgroup. The discussion is limited, however, by the fact that statistical tests for trends 
in effects over time or for differences in trends across target groups have not been riin. 

As shown in the first column for each target group, control group earnings generally 
rose steadily over time in all four target groups irrespective of service strategy. This 
pattern— particularly pronounced for males— reflects the tendency to apply for employment 
and training assistance fi-om JTPA when earnings are unusually low. 

16. These earnings gains in the OJT/JSA subgroup stem in part from the subsidized wages paid 
to JTPA participants in OJT positions^ which averaged between $150 and $190 per OJT participant in 
each target group over the 1 8-month follow-up period. 
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In the classroom training subgroup (top panel of Exhibit 7.6) the most distinct tTend 
in ilacts on earnings is ttot foT adult women. In fact, column 2 displays pTec.sely 
"e w^^ld eTect of a strategy that initially diverts participants from emp^v^ent 
to t.^ Lm so that they may earn more later: significant eammgs losses such as those 
n tTffrqu^r afteT random assignment that turned into significant earnings gams 
by t ^^o^uarters of the follow-up period. But the time trends ^f^^^^^^ 
tiinSg subgroups of the other three target groups were not nearly so clear. Youth 
eSn^ significant earnings losses in the first one to three quarters but no material 
gabs illlater gaiters, while adult men experienced no consistent trend over time. 

Impacts in the OJT/JSA subgroup also followed no obvious trend. ^Jhough Uiey were 
consistently (and sometimes significantly) positive in all ^-f^'J^'^^^^^^^^t LTe^ 
auarters for female youths. For male youths, however, the OJT/JSA service strategy 
'p^u::dLsistentlyWnegative impacts begi^^^^^ 

Lsigmnent, with estimated losses that were statistically significant m two of the six quarters^ 
STe dikerences in impacts between male youths and the other target froups^^^^^^^^ 
Sstableovertime,acomparisonoftherre«rf. in impacts reveals nonotic^^^^ 
actoss urge° groups In oSier words, the OJT/JSA service strategy was either almost 
TdUy benlficial'or-in the case of male youths with a previous arrest-steadily and 
markedly negative in its earnings effects. 

Like the OJT/JSA subgroup, none of the target groups in ih^ other services subgroup 
experienced a distinct trend in impacts on earnings over time, mre is, however, a hin 
ZZ service strategy became less effective over time for the adults, especially adult 
women wh^seem to hfve experienced declining gains after the third quarter after random 
rSr^eT Throughout the follow-up period, female youths appear to have experienced 
negligible impacts, and male youths, markedly negative mipacts. 



General Findings 



To sum .arize JTPA had both its strongest impacts (both favorable and unfavorable) 
LVrmost varied impacts across target groups ^^thin the OmJSA .^-^^^ 
Differences in service receipt across target groups were the least evident for this service 
^S^ ^ measured by the types of JTPA services received, hours of services received 
TcSto the control group, and attainment of a high school credential (agam, relative 
to the control group), lubstantially greater variation in these factors across target groups 
within the other two service strategies did not produce such striking varmtions in impact^ 
suggestingthatvariationsinimpactwerenotproducedbydifferencesinAefactorsconsideed 

here Other potemially important factors may have played a role, however, such as the 
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qualitv' and content of the assistance received or the ability of the service recipients in 
the OJT/JSA subgroup to benefit from the services they received^' 

With one exception noted below, we cannot be sure that either the classroom training 
or other service strategies significantly affected earnings for any Urget group over the 
full 18-month follow-up period. We do know, however, that the classroom training strategy 
reduced earnings initially for at least three of the four Urget groups, with the loss for 
adult women offset to some degree by gains later in the follow-up period. 

The key exception to the generally negligible impact fmdings for the first and third 
service strategies is the finding for male youths in the other services subgroup: a substantial 
loss in earnings that actually surpassed, in percentage terms, the loss susuined by male 
youths in the OJT/JSA subgroup. This loss differed significantly firom the effect of JTPA 
on the other services subgroup of adult women who, in fact, seemed to benefit fi'om the 
strategy early in the follow-up period. Whether this difference should be attributed to 
a different mix of services — relatively more job search assistance for adult women, relatively 
more basic education and high school attainment for male youths — is unclear. It seems 
unlikely that this distinction was decisive, however, since the same service contrast between 
other pairs of adult and youth target groups did not lead to other similariy significant 
variations in impact. 

Taken as a whole, this comparison of earnings and impact patterns across target groups 
is remarkable in its regularity, at least within the range of variation that our data were 
able to detect with confidence. 



Impacts on the Components of Earnings 

Variations in earnings impacts across target groups necessarily stemmed firom impacts 
on one or more of the four components of earnings measured in chapters 4 through 6: 
employment rates, weeks worked if employed, hours worked per week worked, and earnings 
per hour worked. 

Exhibit 7.7 shows a simplified version of the component estimates that appeared in 
earlier chapters. Impacts on the percentage employed, weeks worked if employed, and 
hours worked per week worked are combined here into a single measure — percentage 
effects on hours worked across all sample members and weeks. This component of the 



17. Recall that of all four target groups the OJT/JSA subgroup was by far the most employable 
of the three service strategy subgroups, as measured by the estimated mean earnings of the control group 
over the IS-month follow-up period. 
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Exhibit 7.7 



Percentage Impacts on Total IS^Month Earnings and Selected Earnings 
Components: JTPA Assignees, by Target Croup and Service Strategy 
Subgroup 



Percentage 
impact on: 



AduU 
women 

(I) 



AduU 
men 

(1) 



Female 

youths * 
(3) 



Male 
youths 
(4) 



Earnings per mssignec 
Hours worked per Msigncc 
Earnings per hour worked 
Sample size 



6.2% 
-2.5 
8.9 
2,847 



Classroom training subgroup 

3.5% -9.1% -2.6% 
4.2 -9.7* -2.9 
.0.6 0.6 0.3 
1057 1.045 ^2(5 



Earnings per assignee 
Hours worked per assignee 
Earnings per hour worked 



Sample size 



8.6%** 

5.9* 

2.6 

2.287 



OJT/JSA subgroup 
6.3%* 5.4% 
6.3* -0.8 
0.0 6.2 
2,250 545 



.10.?%* 

"8.3 
-2.1 

6.15 



Earnings per assignee 
Hours worked per assignee 
Earnings per hour worked 
Swnple size 



5.7% 

5.7 

0.0 

L340 



Other services subgroup 
2.1% -2.8% 
1.7 -0.4 
0.4 -2.4 
1M2 710 



-13.3%* 
-9.6 
^.0 

607 



ueaimcol group and conirol group; »ee Appendix D. 
a. Oui-of-ichool youths only. 

t istrrr^Totr.. « .05 « .0. ... uwcu.... or 

Ltiaic^ rig.^f.c«ce were not performed for imp.^ on e.nuDgs per hour worked. 

overall impact on eaimngs captures the employment effects of the P^^°8"^'. ^^^^^^^^ 
from its effects on earnings per hour worked, which are also shown m the exhibu. Because 
we did not test for the statistical significance of either this latter component or the vanat.on 
in impact across target groups, these estimates must be interpreted Nvith great care. 

We focu> first on the sen-ice strategy subgroup with the most obvious variations 
in impacts- Jie OJT/JSA subgroup in the middle panel of the exhibit. Here, ^ m the 
other panels, the first row gives the percentage effects on 18-month earnings, followed 
in the next two rows by impacts on its two components, also m percentage terms. Recall 
from Exhibit 7 6 that, for this service strategy, the estimated effects on earnings were 



18 AS cxDlaincd in earlier chapters, the overall percentage impact on earnings does not exactly 
. LtL ?f re percentage impacts on its components. Close equivalence is evident m all cases 

hrivS:^fth?inS;er^?Lges can be interpreted as the additive decomposition of the overall 

earnings cfrecl into its compon'.nl parts. 



JTPA 1 8-MONTH IMPACTS / TARGET GROUPS AND STUDY SITES • 255 



substantially positive for the first three target groups, and statistically significant for adult 
women and men. Estimates for male youths, however, were large, negative, and statistically 
significant. 

As shown in Exhibit 7.7, the significant earnings gains for adult women recommended 
for OJT/JSA came primarily from a 5.9 percent increase in hours worked, on average, 
although a 2.6 percent increase in average earnings per hour worked also played a role. 
For adult men the OJT/JSA strategy significantly increased earnings through a similar 
6.3 percent gain in hours worked, but did not seem to increase earnings per hour worked. 
In general, then, improvements in the labor market outcomes of adults in the OJT/JSA 
subgroup resulted from more hours of work over the 18-month follow-up period rather 
than from higher earnings per hour. 

A very different pattern emerges from the youth fmdings. Among female youths in 
the OJT/JSA subgroup the program's effect on earnings was almost entirely due to a 
6.2 percent increase in earnings per hour worked. That gain does not imply, however, 
that JTPA necessarily made female youths more productive. It may instead reflect a 
program effect on the subgroup of those who were employed if, for example, additional 
employment was concentrated among female youths with high hourly earnings.*' 

Among male youths in the OJT/JSA subgroup a -2.1 percent drop in earnings per 
hour was accompanied by an 8.3 percent drop in hours worked. Together, these two 
factors produced a large (-10.3 percent) and statistically significant decline in earnings 
over 18 months. 

With the male youth other services subgroup excepted, the other two service strategies- 
classroom training and other services— did not produce statistically significant impacts 
on 18-month earnings, and so the estimates in the top and bottom panels of Exhibit 7.7 
are less interesting. In the classroom training subgroup the largest percentage impacts 
were those on the houily earnings of adult women who worked (up 8.9 percent) and the 
hours worked by female youths (down -9.7 percent). The first finding may be important, 
since it suggests that adult women may have increased their hourly earnings potential 
through additional classroom training.^^ Sizable percentage effects also appeared twice 
v,ithin the other services subgroup: a 5.7 percent increase in the hours worked by adult 

19. In Appendix G v e attempt to control for composition effects of this kind. Although wc find 
no significant program effects on productivity, the analysis was nonexperimental and so its results may 
not be reliable. 

20. Again it is also possible that the gain in earnings per hour worked reflected a change in the 
composition of the subgroup of adult women who were employed. The analysis in Appendix G is not 
fully conclusive on this point. 
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women, and a 9.6 percent decrease in hours worked by male youths, but neither of these 
estimates is statistically significant. 

If indeed they are not just chance patterns in the data, these results would suggest 
that access to classroom training through 3TPA both (1) noticeably improved the tendency 
of adult women-but no other subgroui>-to work iii higher-paid emplo>'ment, and (2) 
substantiallvdivertedfemaleyouths from time thcywouldothei^se have been spent working. 

without any compensating increase in their hourly earnings. The same large diversionarj- 
effect is also evident for male youths ir the OJT/JSA and other services subgroups, although 
the OJT/JSA strategy significantly increased hours worked by adults and potentiaUj' 
increased earnings per hour worked by female youths who worked. Hours worked by 
adult women also increased noticeably in the other services subgroup. 



Impacts on Earnings across Key Subgroups 

In addition to the service strategy subgroups already considered, the three preceding chapters 
examined the effects of JTPA Title II-A on several other subpopulations of the sample. 
For policy purposes the most important of these "key subgroups" that can be defined 
consistently acrosstarget groups arc thosedefined by ethnicity and barriers to employment.- 

Exhibit 7 8 displays estimated program impacts on the 18-month earnings of these 
subgroups in each of the four target groups. Here, rather than showing mean control 
group earnings, the first column for each target group indicates the relative importance 
of each subgroup by giving its sample size (with treatment group and control group members 
combined) Although the estimated impacts on quite a few subgroups were statistical!) 
significant (as indicated by asterisks beside the estimates), none of the estimates m any 
given set of estimates was significantly different from the other estimates m that set or 
panel, as indicated by "n.s." in the F-test rows.^ 



21 In light of our earlier findings for youths, a further subgroup of substantial pol.cy interest .s 
individuals who reported having been arrested between age 16 «,d nmdom assignment. TTie informat.on 
needed vo identify this subgroup of the adult target groups is not availabk. 

22 As explained in the preceding chapters, this pattern of findings means that although we can 
be confident there was a significant impact on the subgroups with an asterisk beside their impact estimates 
we^ot be sure whether Oic impact, on the other subgroups within the panel were significantly different 
from the estimates with the asterisk(s). 
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Ethnic Groups 

Based on the available sample sizes, we can be sure that JTPA affected the earnings 
of whites in two of the four target groups over the 18-month period: it mcreased the 
earnings of white adult women by $723 and reduced the earnings of white male youths 
by $-1,333. We saw this same pattern of results earlier for these two target groups 
as a whole (Exhibit 7.1), which suggests that the overall pattern of targit group impacts 
may be largely attributable to the impacts on whites. 

In light of these results, the main question to ask in looking at the impacts on the 
different ethnic groups is whether the cross-target group pattern for whites overshadowed 
the patterns of effects for the other ethnic groups. The pattern of estimated impacts for 
blacks and Hispanics suggests that this might have been the case, but none of the estimated 
effects on nonwhites was statistically significant. And although the size and signs of 
the estimated impacts for blacks and Hispanics often differed from those for their white 
counterparts, it was only within the target group of adult women that variations in effects 
by ethnic group were close to being statistically significant at conventional levels." 

Thus, the picture for the target groups overall changes only slightly when separated 
by ethnic group: 

• Among adult women in the three main ethnic groups, white women may have 
been the only ones to benefit from access to JTPA. 

• Black male youths may not have experienced the earnings losses that occurred 
for other male youths who had access to JTPA. 

• No ethnic group consistently benefited from access to JTPA at a statistically 
significant level across all Urget groups. 

• Hispanics in all Urget groups may have had their earnings reduced by JTPA, 
but the impact estimates on th.s ethnic group were not sUtistically significant 
and not significantly different from the impacts on the other main ethnic groups 
in each Urget group. 



23 Impacts on adult women difTered across ethnic subgroups at the .20 significance level, but not 
at the more conventional .10 level. As noted in Chapter 4, the negative estimated efTects on Hispanics 
in this target group was concentrated in one or two sites with generally smaller impact estimates for adult 
women overall. 
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Subgroups Facing Selected Barriers to Employment 

The second section of Exhibit 7.8 subdivides the four target groups according to the barriers 
to employment they were facing upon application to JTPA: welfare receipt, limited education, 
and limited recent work experience. Looking at the different barriers one at a time, we 
see first that none of the subgroups receiving cash welfare experienced statistically significant 
effects. Male youth high school dropouts, on the other hand, experienced a significant 
loss in earnings, but in the other target groups high school dropouts were not necessarily 
affected one way or the other. Finally, adult women who worked less than 13 of the 
past 52 weeks benefited from their access to JTPA, while the reverse was true for male 
youths in the same category. Again, however, none of these subgroup estimates differed 
significantly fi'om its complement. 

Because many sample members who were facing one barrier to employment were 
also facing another, the next panel of the exhibit divides the sample by the number, rather 
than the types, of barriers to employment. Here again, variations in the size of the impacts 
across subgroups were not statistically significant for any of the target groups. However, 
large positive impacts occurred for selected adult subgroups: adult women facing none 
or one of the barriers and adult men facing none of the barriers. No striking patterns 
in impacts by number of barriers occurred for the fevnale youth or male youth target 
group. 

In summary, the data identify no subgroups defined based on barriers to employment 
that show consistent evidence of beneficial effects across target groups. Nor do the estimated 
impacts by target group differ significantly with the presence of 'r-.n individual barrier 
or the total number of barriers. 



Impacts on Earnings across Study Sites 

Up to this point all of the impact estimates presented have suminarized the experiences 
of sample members at the 16 study sites as a whole. This section examines the sites 
individually to assess how much program impacts varied across localities. The next section, 
in turn, will investigate likely sources of the variation, namely, selected characteristics 
of program operations, program applicants, and the labor market in the local service delivery 
area (SDA). Since the study was not designed with these objectives in mind, we do 
not expect definitive conclusions here. Even so, anything the analysis can tell us about 
the variations in JTPA impacts across SDAs will add greatly to our knowledge of this 
imporunt policy issue. 
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To increase the size of the research sample in each site, we first combine all adult 
women and men in one analysis and all female and male youths in another. We then 
consider whether patterns of impact vary by gender within each of these age groupings. 
For reasons of confidentiality we do not identify the SDAs by name; instead we consider 
them as a sample of 16 local programs whose characteristics broadly reflect those of 
JTPA programs nationally." 

The following analyses focus on the broadest measure we have of sample members' 
labor market success over the short term: total earnings over the 18-month follow-up 
period. The site-specific impact estimates parallel those presented earlier for the key 
subgroups of JTPA assignees." The analyses in this section do not control for the possible 
determinants of local program impacts, which again are the subject of the next section. 
Instead the analyses here are designed to uncover general patterns of impacts that may 
have been produced by factors that varied ft-om one SDA to another. 

Exhibit 7.9 displays the estimated impacts, by study site, on adult and youth assignees. 
The impacts shown in each column are arrayed in descending order.^« They are noteworthy 
in three main respects. First, the estimated impacts varied considerably for both groups: 
from $1,851 to $-745 for adults, and from $2,566 to $-3,591 for youths. Second, despite 
the broad distribution of impacts within both ranges, the impacts were positive for adults 
in all but three sites and negative for youths in all but four sites. Tliird, and as a result 
of these overall patterns, the estimated impact on the 18-month earnings of adults was 
larger than that of youths at all points on the distribution except the top. 

A considerable degree of uncerUinty applies to these estimates, however. Even after 
combining the adult women and men in one group and the female and male youth ui 
another, sample sizes in individual sites are much smaller on average than for most of 
the subgroups considered elsewhere in this report." Hence, chance variations in the data 
play a bigger role in these estimates than elsewhere. We consider th.s role of chance by 
testing whether the most striking pattern in the exhibit— large differences in estimated impacts 
across sites— reflects real differences in impact across sites or simply chance variations in the 

24 Sec Doolittle (forthcoming) for a comparison of the study sites and SDAs nationally in terms 
of local enviromnenul and program characteristics. Appendix B in the present report compares in a similar 
fashion the baseline characteristics and JTPA program experiences of our 18-month study sample and 
a nationally reprcsenUtive sample of JTPA participants. 

25. Appendix D describes the methodology used to obuin and test the site-specific impact estimates. 

26. Only 15 entries appear in the youth column of Exhibit 7.9 because one site did not include 
youths in its study sample. 

27 Sample sizes are not shown in the exhibits to protect the identity of the sites. For the 16 
SDAs in the adult study and the 15 SDAs in the youth study, sample sizes average 405 for adult women, 
276 for adult men, 153 for female youths, and 117 for male youths. 
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Exhibit 7. 9 Impacts on Total 28-Sionlh Earnings: Adult and Out-of-School 


Youth JTPA Assignees, by Study Site 




Rank of site, bv size 


Impact per assignee. 


in J 


of impact on adults 


Adults 


loutns 


or youths 


0) 


(7) 


1 


$ 1.851*** 


$ 2.566* 


2 


1.332 


339 


3 


1.076 


306 


4 


1.066 


15 


5 






6 


657 


-34 


7 


620 


-239 


8 


616 


-315 


9 


507 


-653 


10 


444 


-707 


11 


414 


-1.093 


12 


394 


-1.268 


13 


55 


-1.372 


14 


-323 


-1,483* 


15 


-585 


-3.591** 


16 


-745 




F-lest, difference junong sites 


n.s. 


n.s. 



Soiircc*: Eiti mate* bued oa Ftnt Follow-up Survey respoam. llie caumuc* for adult woidcs 
are alao basttl on caLnuD{s daU from stale UI a{enciM. 

Noics: The adult awJ youth tistinft io each row do not Dooessarily refer to the same «lc. The 

Mudy was limiuid to 15 titei for yo;^L For the ettiraatioD procedure, tee Appendix D. 

• Statitiically Mtnificai* al the .10 level, •* at the .05 level, **• at the .01 level (F-4«l or iwo- 

tailcd i-tcst); *D.t.* means the P-ieat for the diffeitace ia impacu amon£ the ttudy sites is not 

statistically sigaificaat. 



data. As shown in the last row of the exhibit, statistical tests for differences in impact 
across sites indicate that the site-specific estimates are not significantly different from 
one another for either the adult or the youth sample.^ Thus, on the basis of these data, 
we cannot rule out the possibility that the impact of the program was the same in all 
sites and that the estimates differ purely by chance. 



28. The tests fail to reject the joint hypothesis that impact was the same in all 15 or 16 sites considered. 
These tests are logically equivalent to separate tests that individual sites differ from the average of all 
other sites. These tests, like all others in the report, undersute somewhat our uncertainty about program 
effects in relation to the national JTPA program, since they arc not based on a probability sample of 
the nation. This limiuiion is especially important in this section when considering patterns of impact 
across sites, since it was in choosing 16 specific sites for the study on a non-sutistical basis that national 
representati veness was lost. (As explained earlier, it was not possible to select sites— which had to voluntarily 
agree to the use of rtindom assignment— on a suict statistical basis, despite major attempts to do so.) 
As statements about these particular sites, however, the significance levels shown here (and elsewhere 
m the report) accurately portray our confidence in the findings. 
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Nor can we be sure that pTOgrair. effects differed from zero in most of the mdividua! 
sites examined. Exhibit 7.9 shows that 2 of the 16 site-specific impact estimates for 
adults are sutistically significant, as are 3 of the 15 site-specific impact estimates for 
youths While not large, this total-5 significant impaa cstmiates among 31 tests- 
exceeds the number expected by chance alone-3-using tests that produce sigmfican 
findings 1 time in iO when no impacts occur. Hence, we would expect that at least 
some of the significant results in the exhibit signal local JTPA programs with real earnings 
impacts Most notable is the $2,566 significant increase in earnings for youths in one 
site a finding which differs strikingly from those seen earlier for youths m general and 
those shown here for 14 other sites. Evidently, one site considerably enhanced youtii 
earnings even as the sites as a group reduced or left unchanged those earnings for the 
first 18 months of the follow-up period. 

Analyses identical to those in Exhibit 7.9 were used to calculate site-level impacts 
for adult women and adult men separately, and for female you^ ^^ mle youths separate >s 
^ show^ in Exhibit 7.10. Here again, estimated effects differ widely but ms.gnificantly 
across sites, leaving it uncertain as to whether true ^^P^^^/^ff^^^^ f V^"" f"'' 
target group. Positive estimated effects for adult women and men, and negative cstii^ted 
effects for male youths, do appear to be widespread, however. Significant positive effec^ 
appear in selected sites for the two adult Urget groups and significant negative effects 
for male youths. StiU, the total number of significant findings-5 in 62 tests-~is less 
than the number expeaed by chance (6). Hence, we cannot be sure that real effects 
occurred in any particular site. 

A final question to be addressed here is whether local programs that worked best 
to increase earnings in one target group did the same for the other target groups. While 
this seems intuitively plausible, the reverse pattern could also hold if SDAs tended to 
focus their efforts on one or two target groups to the exclusion of the others, or if policies 
and approaches that worked well for some Urget groups did not do so others^ io 
examine these possibilities, we rearranged the site-specific impact estmiates from Exhibits 
7 9 and 7 10 into the three graphs shown in Exhibit 7.11. The first of these graphs 
displays the all-adult and all-youth findings from Exhibit 7.9 in graphic form, starting 
at the'top with the site with the largest estimated impact on adults and continuing dowTi 
the page to the site with the smallest (i.e., most negative) estimated impact for adults. 
The size of each impact estimate is indicated by the length of the horizontal hne segments, 
with the adult impact estimate shaded white and the youth impact estimate shaded gray 
for each site. Positive impact estimates extend to the right of center (the $0 point) and 
negative impact estimates extend to the left.^ 

29. Only 15 sites appear in Exhibit 7.11 because one site did not include youths in its study sample. 

30 All three graphs in Exhibit 7.11 use the same left-t«vright scaling. Hence the length of any 
two line segmcnts-oie size of any t^^.o impact estimates-is comparable across graphs, rather than thc.r 
rank order. 
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Exhibit 7. JO Impacts on Total IS^Month Earnings: JTPA Assignees, by Target 
Group and Study Site 



Rank of site, by size 
of impact on the 
target group 



Adult 
women 

(1) 



Adult 
men 

(2) 



Female 
youths " 
(3) 



Male 
youths 
(4) 



1 $1,738 

2 1.583 

3 1.249* 

4 1,162" 

5 1.058 

6 931 

7 587 

8 474 

9 390 

10 254 

11 217 

12 127 

13 111 

14 -352 

15 -363 

16 -655 
F-tcst, difference among sites 



$ 3,050* 
1,487 
■ 1,420 
1,123 
1,064 
984 
910 
811 
557 
478 
191 



-423 
-1,084 
-1,124 
-1,718 

n.s. 



$1,473 
1,105 
963 
830 
805 
292 
-115 
-368 
^32 
-475 
-632 
-811 
-914 
-1,447 
-2*496 

n.s. 



S 4.424 
1,597 
506 
335 

-149 

-656 
-1.165 
-1.199 
-1,604 
-1,654 
-1,969 
-2,249 
-2,374* 
-2.643 
-6,581** 

n.s. 



Sources: Eiiim.ies based on F,nt Follow-up Survey mpo««. Tl>c esiimiie. for .duU women «t also based on 

lo 1 5 siic* foe youihi. For Ihe eaimation procedure, see Appendix D. 

Sr«!:i;rS::ri'.hc .O .cvC. - ..Uk .OS .evd. ... . .0. Uvc. (F.«. or .w.uV.ed MesO 

•n.i.- m«ni the F-te* for the difference in invicu among the «udy site* it not ««iiKic*lly t.pufic.nt. 

Visually, it appears that SDAs with rclaUvcly positive impacts on adults also tended 
to have relatively positive (i.e., less negative) imparts on youths. A more formal test 
of this hypothesis-that some sites did relatively well with both adults and youths while 
others did poorly in both instances— is provided by the correlation coefficients that appear 
just below the graph.'' These coefficients show that, in both rank order and dollar magmtude, 
the adult and youth findings correlate positively across sites, with correlation coefficients 
of 40 and .53, respertively, on a scale of -1 (perfert inverse correlation) to 1 (perfect 
positive correlation). If this panem is not due simply to chance variations in the data. 

31 The first of these coefficients, the rank-order correlation coefficient between the adult and youth 
impact estimates, shows the degree to which sites occupy similar positions In the two rankings. A value 
of 1 for this measure represents perfect correspondence between the two rankings. . value of 0 no pattern 
at all and a value of -1 an exact reversal in the two patterns. The second measure, the dollar-value 
corrclat.on cocffi.cient. follows the same scale but is br .ed on the dollar values of the impact estimates 
Sec Appendix D for the formulas used in calculating each of these measures. 
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Exhibit 7.11 Correlation of Impacts on Total 18-Month Earnings across Target 
Croups, by Study Siu 




SO 



-4000 



-2000 0 2000 

□ Adultr M Youths' 

Rank order correlation coefficient = 0.40 

Dollar value correlation coefficient = 0.53 



4000 




-2000 0 2000 -WOO 

□ Adult women ^ Adult men 

Rank order correlation coefficient = 0. 1 1 
Dollar value correlation coefficient = 0.31 



SO 




Rank order correlation coefficient =0.19 □ Fenmle youths- ^ Male youths- 

Dollar value correlation coefficient = 0.62 

Source.: E«im..e.ofimp^uper«ign.e>don.r.,b.^onF.r..Follow.upSurvcyre^n.c..Thee.^^ 

lined fjrn in the key to etch graph. 
a. Oui-of-ftchool youths only. 



3.iO 



JTPA 18.M0NTH IMPACTS / TARGET GROUPS AJ<D STUDY SITE^ • 265 



one can conclude that, among the 15 SDAs exanained here, a local program's effeciiveness 
in serving adults was positively related to its effectiveness in serving youths. 

Similar graphs comparing adult women with adult men, and female youths with male 
youths, also appear in Exhibit 7.1 1.^^ For adults, there is only a weak positive relationship 
between a site's relative effectiveness serving women and its relative effeciiveness serving 
men. Hence, knowing how well a site did in serving women tells us very little about 
how well it did in serving men. A somewhat stronger positive correlation exists between 
site-specific impact estimates for female youths and male youths. Hence, the SDAs that 
participated in the experiment achieved more consistent results across genders for youths 
than for adults. 

To summarize these results, we cannot be sure that program effects differed across 
sites for any of the four target groups considered. Nor can we conclude that sites that 
produced the largest impacts for females did the same for males, although a consistent 
pattern of this sort does emerge when comparing all adults with all youths. In general, 
the overall lack of significant patterns in impact across sites may result from the small 
number of individuals analyzed at each site and the resulting uncertainty about true effects. 
Just as easily, it could mean that the 16 SDAs studied really did differ little in their 
impacts but appear to differ in our data due to chance variations alone. 

The Influence of Selected Site and Baseline Characteristics 

on Program Imoacts 

A final component of the analysis seeks to explain patterns in estimated impacts by site 
in terms of local program characteristics, local labor market conditions, and participant 
baseline characteristics. The principal objective here is to understand how program operating 
decisions affected impact, in order to identify the local approaches that produced the best 
results. We have already seen (in the subgroup analysis) that participant baseline char- 
acteristics can also influence program impacts. Hence, we will need to control for baseline 
factors when measuring the influence of local program characteristics on program success. 
The influence of external faaors such as local labor market conditions will also need 
to be controlled for in the analysis.^^ 

32. Note that only 15 sites are included in the youth graph since one site did not include >-oulhs 
in its experiment. 

33. Unless we control for these factors, dificrcnccs in baseline characteristics and local conditions 
could create the appearance that some SDAs opcTBtcd their programs more effectively than others when, 
in fact, they merely faced a different set of initial conditions. Alternatively, differences in external conditions 
could mask important variations in operational effectiveness that would otherwise be apparent, as would 
happen, for example, if more effective SDAs faced atypically unfavorable labor market conditions. 
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The Site-specific impact estimates shown earlier reflect the net effect of all of these 
factors taken together. Since those effects did not differ significantly across sues we 
do not expect many of the site-level factors tested here to have stat.st.ca ly s.gn.f.can 
influences when examined separately. Still, it is only in looking for the local determmant 
of program impact that we have any chance of providing practical guidehnes for improvmg 
local programs. 



Possible Determinants of Impact 

In searching for the determinants of local JTPA impact, we begin by observing that 
measured program impact in any site depends on: 

. Decisions made by the SDA in serving clients during the study period; 

. The conditions faced by those clients when they entered the labor force, including 
characteristics of the local labor market and of the non-JTPA employment 
and training services available in the community (which should have influenced 
primarily earnings levels in the control group against which mipacts are 
measured); 

• The types of clients accepted for JTPA services; and 

. The proportion of the client group studied— the assignee sample-that actually 
received JTPA services. 

The potential for each of these factors to influence impact is fairly obvious. Some 
individuals may be better able to benefit fi-om JTPA services than others, dependmg on 
their background characteristics, family situation, and prior educational and employment 
experiences. Similarly, the strength of the local labor market and the availability of services 
outside of JTPA may condition how much the program can do to mcrease participant 
earnings relative to what they would have earned without JTPA. Also, the type, quality, 
and duration ofserviccsprovided through JTPA-detenrnnedbySDAoperatingdecisions- 

will almost certainly influence program impacts. Finally, the greater the share of assignees 
served in an SDA. the larger the program impacts should be on the average assignee. 

, To identify the determinants of program effectiveness at the local level, we estimate 
proeram impact as a function of these four sets of factors-participant baseline char- 
acteristics, local enviromnent, program approach, and the assignee enrollment «Jte-using 
data on individual sample members. Hie specific variables used are described below, 
beginning with the program measures on which the analysis focuses. These variables 
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were chosen from a longer list of candidate measures as tiie ones most likely to (1) produce 
important variations in impact and to (2) vauy substantially across individuals, local program 
offices, and/or entire SDAs, 

Three aspects of local program operations are included in the model: 

• The percentage of assignees in the site recommended for the three different 
sennet strategies — classroom training, OJT/JSA, and other services. Two 
sets of variables are used to represent the service strategy mix in a 
site: indicators of whether an individual was recommended for classroom 
training, OJT/JSA, or other services, and indicators of the percentage of 
assignees recommended to each service strategy for the site as a ^ole. Both 
sets of variables fall under the control of local program operators in deciding 
what service strategy will woric best for individual clients and in determining 
the overall service strategy orientation of their programs. This latter &ctor — 
overall program orientation — could influence impacts in ways not reflected 
by service strategy decisions for individual clients and is included in the nuxlel 
separately for that reason. For example, SDAs that focused more on other 
services and less on traditional classroom and on*the*job training may have 
provided more innovative and individualized services in all service strategy 
subgroups. 

• TTie percentage of enrollees in the site facing two or more barriers to 
employment at baseline. After controlling for die baseline characteristics of 
individual assignees (see below), this measure reflects an SDA's willingness 
and ability to concentrate its services on the more disadvantaged clients among 
those in its assignee pool. 

• The percentage of training dollars spent under performance-based contracts 
in JTPA program year 1988, the midpoint of fbc service period for enrollees 
in the 18^nonth study sample. Sites that reimburse outside organizations 
for providing training services only if certain pre-specified performance levels 
are attained (e.g., placement in employment of a certain proportion of trainees) 
may have very different impacts than those who pay regardless of the training 
ou^' 3me. 

All of these variables arc measured separately for adults and youths and — except for 
the percentage of performance-based contracts — for females and males within the adult 
and youth groups. 
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Two measures of the local labor market are included in the model: 

. Urban/rural location (coded separately for each local office in the 16 SDAs, 
based on population density and the characteristics of the office setting). 

• Local unemployment rate (a monthly average for the period July 1987 to 
June 1990). 

Weuse several variables to control for differences mparticipant baseline characteristics 
across sites, each one measured at the individual level for each assignee and control group 
member in the sample:** 

. Age at baseline (variables distinguishing ages 22-29, 30-44, 45-54, and 55 
and over for adults, and ages 16-19 and 20-21 for youths); 

• Gender: 

• Ethnicity (variables distmguishing whites, blacks, Hispanics, and others for 
adults, and blacks, Hispanics, and others— including whites— for youths); and 

• Barriers to employment at baseline (separate indicators for individuals who 
had neither a high school diploma nor a GED and— for adults— individuals 
who worked less than 13 weeks in the previous year and individuals receiving 
cash welfare at ^plication to JTPA). 

Finally, to adjust for differences in the extent of services received by the assignees 
in the analysis sample, wc add a variable measuring: 

• The percentage of assignees in each site enrolled in JTPA during the 18- 
month follow-up period. 



Ln<lTATIONS OF THE ANALYSIS 

Before turning to results, wc should note three limitations that apply to any attempt to 
attribute cross-site variations in impact to specific causal factors. Each of these limitaUons 



34 Althou^ it subsuntitlly innucnced the size of program impKls for male youlhs. we do not 
include prior arrest record as a baseline variable in the model. Dau on this charactenst.c are not available 
for adults. 
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Stems from a different source which, while independent in origin, have a cumulative effect 
on the overall reliability of the analysis: 

• We can observe the influence of any particular local program characteristic 
only in combination with all of the other local program characteristics in the 
site. As a result, we must rely on a non-experimental model to separate out 
the role of each factor. Where those faaors correlate with one another, or 
have non-additive effects, this separation will be only approximate. As a 
result, some portion of the influence of one factor may be mistakenly attributed 
to another factor. 

• We have only 16 sites to work with, compared to dozens of local program 
characteristics that might conceivably influence program success. Because 
of this constraint, we can analyze only a handful of the candidate measures 
of interest, potentially omitting some that may strongly influence program 
impact.^^ Not only does this create the potential of overlooking an important 
program characteristic, it may also lead us to falsely attribute the influence 
of an omitted factor to included variables that happen to correlate with it. 

• Finally, even if we manage to include all important program measures in the 
model and avoid confounding one with another, our ability to confidently quantify 
the influence of any one factor is constrained by the small samples sizes available 
in each site. To trace cross-site variations in impact to specific causal fsictors, 
we must first be able to measure those variations with confidence. We have 
seen already how imprecise the site-specific impact estimates are because of 
their small sample sizes. 

Despite these limitations, the model used here provides the best information available 
on the sources of variation in local program effectiveness. Its strengths are that, in contrast 
to many ; ^vious studies of this sort, it: 

• Uses a formal standard of evidence (statistical significance) to determine which 
factors are causal rather than coincidental; 



35. Considering more variables by csiimaling the model on one set of candidate measures and then 
another would give misleading results. With repealed attempts, one or more versions of the model would 
almost certainly give an apparently complete and compelling explanation of the patterns in the data. Bui 
because it emerged from a 'nrial and error" process, what it would really represent is ihc **lriai" that 
sooner or later was bound to fit the data well by chance alone once enough variations were U-ied. There 
is no reason to believe that a model selected on this basis correctly reflects the real sources of impact 
variation or has any generality for SDAs outside of the sample from which il was derived. 
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Considers several fectors at once in order to control for locd labor market 
and participant characteristics when measuring the influence of local program 
characteristics; and 

Chooses in advance the factors to be considered, rather than letting potentially 
spurious patterns in the data dictate which fectors are considered or advanced 
as causal. 



Findings 



The findings from this model appear in Exhibit 7.12 for adults and youths separatel>^ 
Envies in the exhibit indicate which of the above variables are <^«nnted to have had 
a statistically significant influence on the size of program unpact m the study s.tes. holdmg 
CO sSt the other variables in the model. Significant influences are nurked by astensks 
Zd discussed in detail below. Influences that are not statistically s.gmficant are md.cated 
by "n.s." in the exhibit. 

For adults, none of the program operating characteristics considered here significantly 
influenced program impacts. Moreover, the one significant mfluence sho^vn-smalle^ 
nnpacts in sites that served a greater share of cash welfere rec.p.ents-emerged from 
a set of 15 tests and may reflect nothing more than chance vanations m the data. Separate 
analyses of adult women and adult men (not shown) produced similar results. 

Hie results for youth are more interesting. Here, the operating characteristics of 
local JTP> programs as a group-and the mix of service strategies recommended m 
particular-had a significant influence on the size of program impacts." Separate analyses 
(not shown) indicate that this pattern is confined to the male youth target group, since 
none of the variables in the model significantly influenced impacts on young women. 

36 AS before, the youth analysis is confined to the 15 SDAs that included youth in the.r experiments. 

37 -nie cash welfare variable was again statistically significant at the . 1 0 level and negatively infiucnced 
imJi on ad^t worneT Hie only oth?r variable to significanUy infiucnce impacts for either adult women 
oT^du^rn iT l J^^^^^ rate, which had a significant (at the .10 level) negative effect 
on the size of program impacts on women. 

38 The significance of these infiuences was determined by first testing and 'he joint h^^thes^^^ 
that Spa^ doeTnot depend on any of the program operating charaC istics included m the m«iel a d 
,h.n hv tesiini. and reiectine the narrower joim hypothesis that impact does not depend on what -rv.ce 
'^:^S.::T^:^^.TZ ^^ individual 'assignees or on the mix of service strategics employed by a 



sue overall. 
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For young men, however, the mix of service strategies does correlate quite convincingh' 
with' the size of program impact, all other things equal?^ 

Tne nature of this correlation is quite complex, however. An examination of the 
mdividual regression coefficients (not shown) suggests that, among the 15 SDAs examined, 
those SDAs that focused their programs most squarely on the other services service strategy 
achieved the best (i.e., the least negative) overall impacts. Once the general orientation 
of a site is taken into account, however, recommendation to the other services strateg>' 
is less likely to produce favorable results for an individual assignee than is recommendation 
to classroom training or OJT/JSA. 

Interpreting this pattern is not straight-forward. It may reflect in part the distribution 
of male youths with prior arrest records (whose estimated effects were much more negative 
than those of other male youths) across sites and service strategies. We plan to add 
the arrest variable to the model to sort out these relationships as part of the forthcoming 
30-month follow-up report. 

Exhibit 7.12 identifies one other source of impact variation for youths: residence 
in an urban versus rural location. Program impacts on 18-month earnings are estimated 
to have been $1,207 more negative for those youths who applied to JTPA at an urban 
office, all other things equal (not shown). This pattern is particularly striking among 
male youths, where there was an estimated urban/rural difference in impact of -$2,600. 
This may in part reflect the distribution of male youths with prior arrest records between 
urban and rural sites. 

With these exceptions, none of the sources of impact variation across sites could be 
identified at conventional levels of statistical confidence. This is not surprising given 
the earlier conclusion that large measured variations in impact across sites may reflect 
nothing more than chance variations in the data— a consideration that should further caution 
against making too much of th'^ statistically significant findings which were discovered. 
The major message of the site-level analysis, then, is that the available information cannot 
reliably distinguish between random "noise" in the data and true patterns of program 
impact across sites. If real impacts varied across the study sites and/or specific factors 
systematically influenced those impacts at the local level, the data provided by this study 
generally are not capable of measuring those variations with confidence. This, too, is 
unsurprising given that the study was not designed with this purpose in mind. 



39. Joint tests of all program operating characteristics taken together and of the service sirategv 
variables alone give statistically significant results at the .05 and .01 levels, respectively, for male youths 



ERIC 
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Exhibit 7.12 Influence ofSeleaed Site and Baseline Charaaeristics . and Sets oj 

Characteristics, on the Size of the Jmpaa on Total IS^Month Earnings:. 
Adult and Out-of-School Youth JTPA Assignees 



Impaa per assignee, in S 

Adidts Youths 

Set/Characteristic W 

Operating characteristics of the program 
(6 variables) 

Service strategies recommended (4 variables) n.s, 

% of enrollees facing two or more barriers to 
employment 

% of training dollars spent under performance- 
based contracts 
Characteristics of the local environment 

(2 variables) 

Urban/rural location ^-^ 
Local unemployment rate 
Baseline characteristics of assignees 
(10 variables for adults, 5 for youths) n.s. 
Gender 

Ethnicity (3 variables for adults, 2 for youths) 
Barriers to employment (3 variables for adults) n.s. 
Receiving cash welfare ^ * 
No high school diploma or GED certificate n.s. 
Worked less than 13 weeks in past 12 months 
Age (3 variables for adults) 
Enrollment rate of ^ignees at the site n^. n.s. 

Sample size ^£f^f 4P48__ 

Sources: Ettimaics based on Hni FoUow-up Survey responses. The estimates for adult women .re .iso 
based on earnings dau from ttate UI agencies. 

Notes- The influence is measured by the regression of individual sample members* 18-month camrngs 
on site and assignees* baseline characteristics, an indicator of membership in the treatment group, and the 
interaction of the treatment group indicator and the site and baseline characleriaics listed. Enincs mdicaie 
the statistical significance of the interaction terms. For the estimation procedure, sec Appendix D. 

a. AFDC. General Assistance, or other welfare except food stamps. 

b. Not tested in the regression. . 

• Statiftically significani at the .10 level. " at the .05 level. at the .01 level (F-test or two-u.lcd t-tesi); 
•n.s.* means the influence of the characUriaic on the impact is not sutistically significant. 



n^. n.s. 

n.s. 

ns, n.s. 
n.s. n.s. 

n.s. 

n^, n-S- 
n^. ii-s- 
n.s. 
_b 

n.s. 
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A Summary of Cross-Target Group and Cross-Site Compansons 

The results presented in this chapter focus on the nature and origins of variation in impacts 
across taiget groups and across sites. By cataloging these variations in relation to other measured 
differences between target groups and sites, wc take the first tentative step toward understanding 
the nature of the intact findings presented elsewhere in the report. 



Target Groups 

Taken as a whole, the effects of JTPA on the four target groups differ substantially. 
The major contrast concerns differences between adults and youths, although systematic 
male/female differences are also evident, especially for youth. In brief, the four target 
groups can be summarized as follows: 

• Adult women most clearly benefited firom JTPA participation. This benefit 
came late in the follow-up period if recommended for classroom training, 
throughout the period if recommended for OJT/JSA, and cariy in the period 
ifrecommended for other services. Usually, these earnings gains were attained 
by working more hours rather than earning more per hour, although the reverse 
was true of the classroom training subgroup. Additional services received 
through access to JTPA concentrated most heavily on classroom training in 
occupational skills. 

• The evidence of positive impacts on adult men generally parallels that for 
adult women but is not quite as strong. Men differed firom women in several 
important respects, however. They earned substantially more absent JTPA 
and were more likely to obtain on-the-job training because of JTPA. And 
unlike adult women, classroom training did not increase adult men^s earnings 
by allowing them to earn more per hour. 

• Unlike the two adult groups, yem^^/e youths seem not to have benefited from 
JTPA participation in general. They earned less absent the program than 
any other target group and tended to receive additional hours of classroom 
training (both in occupational skills and basic education) when allowed access 
to JTPA. The program noticeably affected their earnings in two instances: 
early losses in earnings within the classroom training subgroup (where hours 
worked declined substantially) and substantial but statistically insignificant 
earnings gains in the OJT/JSA subgroup (where earnings per hour rose 
substantially). 
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Male vouths were the one urget group which experienced a negative impact 
from 'the participation in JTPA, as measured by the follow-up survey data, 
lecially those with records of prior arrests.^ Large, sustained losses m 
earnings t>pified the male youth experience w^th the OJT/JSA serv.ce strategy 
and the other services strategy (which provided primarily miscellaneous services 
and basic education). TT^eselossesresultedprimarilyfromsubstannaldecr^^^ 

in hours worked. Male youths were more often recommended for the other 
services subgroup than female youths but othenvise received very similar 



services. 



Sites 



A comparison of estimated impacts across sites revealed little about the source and nature 
of JTPA impacts in part because of the small samples available m each site. In most 
Is at^^s to mA is estimated to have increased the earnings of adult assignees and 
Suc^Jie ^mings of youths, although these patterns are not statistically significant 
L t o^ st . ^re is also som^ suggesfon in the data that shcs which achieved relatively 
more favorable outcomes for adults also tended to do so for youths. 

The degree of variation in impact across sites is uncertain. For no target group was 
this^ariat on statistically significant, and only rarely could it be traced to specific site- 
tel characteristics. None of the factors hypothesized to influence earnings gams at the 
loci e^-program operating features (e.g., the mix of service strategies employ d). 
o^a Sor m'arket conditions (e.g., unemplojonent rate), and participant char— s 
e g barriers to employmr.it>-were found to have done so for adult women, adult men 
or female youths. LZo local factors found to significantly influence program unpact 
?or maJeyouths-the mix of service strategies employed by the local SDA and urban^ 
"raHetting^may no longer do so once the distribution of male youths with prior arrest 
records is equalized across sites and service strategies in a later analysis. 



Summary 

The adult/youth contrast which predominates these findings may chan^ie once longer- 
term follow-up data become available, or once program costs are considered in the final 
beTeflt "o^ analysis. Either way, fiiture research should seek a dearer understanding 

^c^ A no.<.H ,n ChaDter 6 a secondary data source (unemployment insurance wage records) does 

not :Lt^^XcS:L^L.^. ytS.s •I.e contrast in these two findings wH. be .nvcst.gated 
further in Ihc final report for the study. 
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of why these short-run differences in impact arose. Three preliminar>' pieces of eWdence 
are available already: 

• First, it is possible that the adult/youth differences in impact are the result 
of age alone. As discussed in Chapter 6, impacts on youths do not differ 
significantly from what is expected for young assignees given the downward 
trend in impacts that emerges when examining successively younger subgroups 
of adults. For men, however, estimated impacts for 16- to 21-year-olds he 
considerably below this trend line— perhaps due to large negative effects on 
male youths with prior arrest records— though the departure from the trend 
is not statistically significant. Hence, we cannot be sure that the adult and 
youth programs differed systematically in their effectiveness for either gender. 

• S econd, to the extent that program differences do account for observed differences 
in impact between adults and youths, they do so without creating large measured 
differences in service receipt. In terms of recommended services, the most 
striking differences distinguish women from men, not adults from youth. 
Moreover, differences in impact between adults and youth are just as evident 
when sample members are grouped by service strategy as for Urget groups 
as a whole. In the one service strategy subgroup where the impact contrast 
is sharpest— OJT/JS A— there is essentially no difference by age in the additional 
hours and types of services obtained through access to JTPA. Differences 
for the other two service strategies are more apparent, but still modest. 

• Third, it is possible that earnings reductions for male youths resulted in part 
from Ae mix of service strategies employed. Service strategies relate to program 
impacts in a complex way, however, and may be confounded with differences 
in impacts for youths with and without records of prior arrest. 

Finally, it is important to note that, even if the adult and youth programs differed 
in important and perhaps unmeasured ways, we should not conclude that JTPA is necessarily 
as capable of helping youths as it is adults. As noted earlier, youths have somewhat 
different baseline characteristics and earnings potential absent the program, factors that 
may or may not limit their ability to benefit from a given JTPA treatment. So we cannot 
be s ure that equalizing treatment would necessarily equalize effects. Thus, while the program 
has clearly found a way to improve the short-run labor market outcomes of adults, we 
do not yet know how it might move closer to that objective for female youths and— 
in the case where change is most clearly needed— male youths. 



Appendix A 

A Statistical Comparison of 
Treatment-Control Group Differences in 
Baseline Characteristics 



THIS appendix describes how we used a multivariate discriminant analysis to 
determine the comparability of the treatment group and the control group in the 18- 
month study sample, using the data on baseline characteristics that were presented in 
Chapter 3 (exnibits 3.13 and 3.14). Three potential probl^ns must be addressed in 
conducting such an analysis: 

• A mS^i/e comparisons problem arises when one tests many hypotheses. 
Doing so IS likely to produce by chance significant test statistics for some 
of the hypom^^r^e^gn^when the populations being compared are identical. 
For example, when testing differences for statistical significance at the .05 
level, 1 out of 20 independent tests will be significant when in fact no real 
differences exist. 

• Interdependencies (correlations) among the different test statistics occur 
because characteristics are usually distributed across individuals in patterns, 
not randomly. For example, income and education are often correlated. 
Hence, a test of differences in income between a treatment group and a control 
group would not be independent of a test that compared the educational levels of 
the tWv^ groups. 

• Missing data are virtually inevitable in any large-scale empirical study. 
Here the problem is that different data usually are missing for different 
sample members. In the model described below, we avoided the problems 
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of multiple comparisons and interdq)endencies among the separate 
comparisons by "pooling" the analyses of separate baseline characteristics. 
In addition, to avoid deleting sample members from the analysis when data 
were missing for some, but not all, of the baseline characteristics measured, 
we specified the statistical model used to compare treatment group and 
control group members to account for patterns of missing data. 



The Basic Statistical Model 

A standard form of the multivariate discriminant model is based on the assumption that 
the characteristics in two populations (in this case, the populations from which the 
treatment and control groups were drawn) have a joint normal distribution in both 
populations, with a common covariance matrix but different mean values. Under this 
assumption we can generate a discriminant function for the probability that an observation 
(sample member) with given characteristics comes from one of the two populations. 
Furthermore, as developed by Fisher (1938), we can test the hypothesis that the measured 
characteristics do not discriminate between the two populations (in other words, that the 
mean values for the two populations are the same) by using the usual composite test for 
a set of estimated coefficients from an ordinary least squares regression of the form: 

(Al) TREATMENT^ = n + Yi^k^^^h * 

k 



where 



TREATMENTi 

CHAR^ 
a 



1 for treatment group members, and 0 for control 
group members; 

the value of baseline characteristic k: 
an intercept; and 
a random error. 



To summarize the overall difference between the sets of baseline characteristics of 
the treatment group and the control group, we used the produced from estimating a 
model of this type by ordinary least squares. To determine the statistical significance of 
this overall difference, we used a standard F-test for the hypothesis that the coefficients 
of the characteristics in Equation Al were all zero. This test is mathematically equivalent 
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to the test for a multivariate discriminant model that indicates the statistical significance 
of the difference between two multivariate normal distributions.' 

The foregoing procedure enabled us to account for interdependences among the 
treatment-control group comparisons for different baseline characteristics and to solve the 
multiple comparisons problem that arises when making comparisons of many different 
characteristics by testing a single composite hypothesis. 

Interdependences among the diflFerent baseline characteristics were accounted for by 
including them together on the right-hand side of a multivariate model and conducting 
a joint test of their overall significance. In such a model correlations among right-hand- 
side variables are reflected in the variance-covariance matrix that is the basis for 
parameter estimates and test statistics. Thus, for example, the measure we used to 
summarize treatment-control group differences evaluates the set of baseline characteristics 
as a group and nets out overlapping effects of separate variables. 

The single joint test we used to determine the statistical significance of treatment- 
control group differences for each target grorn in eflfect summarized the results of many 
comparisons into a single one. This reduced the chance of observing random, but 
seemingly significant, differences, as tends to happen when employing a larger number 
of tests. 



Extension of the Basic Model 

An extension of the model was necessary to account for panial missing data. 
Specifically, we added a series of dummy variables on the right-hand side to indicate the 
presence or absence of data for each baseline characteristic. These dummy variables 
were defined as follows: 

DATA^i = 1 if data on baseline characteristic k were available for sample 
member z, and 0 if not. 

Because in some cases two or more baseline charaaeristics were measured using 
responses from related questions on the Background Information Form, the missing data 
patterns for these characteristics were the same. In these cases we therefore used a single 
data availability indicator for each set of baseline characteristics. 



1. Sec Fisher (1938) or, for t simpler discussion, Htggstiom (1983). 



280 • HTA IS-MONTH IMPACTS /TIU£ATM£m'<X)N11lOL GROUP 

A final, related extension of the basic model was to set the value of each baseline 
characteristic, CHAR^, equal to zero for those sanq>le members who were missing data 
on this characteristic. This redefined characteristic variable, CHAR!u, is an interaction 
between the original characteristic variable and its corresponding data availability 
indicator.^ The final multivariate discriminant model was therefore specified as follows: 

(A2) TREATMENT^ ^ a ^ ^b^CHAR^^^ + J^AWj^,. + 

k k 

We used ordinary least squares to estimate this model from data on all members of each 
target group. 



The Analysis and Findings 

To pool the comparison of all the baseline characteristics into one joint test per target 
group, we computed the incremental for the set of characteristics variables, CHAR^i^, 
for that target group. The incremental is the difference between the R^ for the full 
model for the group and the R^ for a nuxiel that omitted the CHAR^^? 

The incremental R^ for the baseline characteristics is a direct measure of the squared 
correlation that will exist between the baseline characteristics and treatment group or 
control group status when these variables are used in the right-hand side of multiple 
regression models for estimating program impacts. This correlation will reduce the 



2. To sec how missing diU indicators and their corresponding baseline characteristics can be interpreted 
in the model, consider an example with one characteristic. Suppose that: 

RACE\ = 1 for whites. 0 for nonwhites, and 0 for sample members with missing data; and 

DATA^ = 1 for sample members with data on this characteristic, and 0 for those with missing data. 

The treaiment-conirol group discriminant model for only this factor would be: 

TREATMENT, = a + i>, RACE'^ + c, DATA^ + 

The coefficient c, in this model measures the difference in missing-data rates for treatment group and 
control group members. The coefficient measures the treatment-control group difference in the avenge 
value of RACE\ for sample members with data on this variable. 

3. This parameter represents the overall difference in baseline characteristics between the treatment and 
control groups, after we controlled for the rates of missing data. 
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precision of program impact estimates. Specifically, minimum detectable effects* will 
be inflated by a factor (7F^, defined as: 



{A3) IF^ = , . 



where is the mcremental for the baseline characteristics and, hence, reflects the 
correlation between these characteristics and a dummy variable that distinguishes 
treatment group members from control group members.^ For example, if the R^ were 
0.01, the minimum detectable effect would be LOOS times what it would have been if the 
R^ had been zero. This would increase a $500 minimum detectable effect to $502.50. 

Exhibit A.l presents the incremental R^ for each target group in the 18-month study 
sample, while Exhibit A.2 lists the variables used in the comparison. As shown in the 
first exhibit, the incremental R^s range from 0.0032 to 0.0128, which implies the 
minimum detectable effects will be inflated by factors ranging from 1.002 to 1.006. In 
other words, the measured differences between the treatment group and control group 
within each target group will reduce the statistical precision of the program impact 
estimates for each target group by a negligible amount. 

A second key finding shown in Exhibit A.l involves the statistical significance of the 
overall differences in baseline characteristics between the treatment and control groups. 
We addressed this issue by computing the F-statistic for each incremental R^ and 
determining the probability of an F-value being equal to or greater than the observed 
value under the null hypothesis of no difference between the baseline characteristics of 
the treatment and control groups. This probability, or p-value, is the lowest test level 
for which the observed differences would be statistically significant. 

The second row of Exhibit A.l displays this p-vdue for each target group. In the 
case of adult women, for example, the difference in basdine characteristics for the 
treatment and control groups is statistically significant at only the .95 level. This implies 



4.1n simplest icrms. a minimum detectable effect is the imtllcfit real program effect thai has a good 
chance of being detected— that is, yielding a finding of a lUtistically significant impact— using the daU 
available. 

S.This suiement reflects the fact that a sundard error of estimate for a regression coefficient is inversely 
proportional to the independent sUndard error of the right-hand-side variable to which it refers, where the 
independent sUndard error is based on the variation in the varuble that is not correlated with any other 
covariates. 
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a 95 percent chance of observing treatment and control group charaaeristics that differ 
by as much as or more than those for the 18-month study sample of adult women, when 
there is no difference, on average, in the population from which they were sampled. The 
corresponding significance levels for adult men, female youths, and male youths were 
0.29, 0.40, and 0.94, respectively. Hence, no treatment and control group differences 
were significant at the conventional 0.10 or 0.05 levels. 

In summary, then, the findings of this statistical analysis indicates virtually no 
difference in measured baseline characteristics between the treatment and control groups 
within each target group. Those differences that were observed are neither substantial 
(as measured by the incremental nor statistically significant (as measured by thep- 
value for 'Jie incremental R^. Our findings are therefore in accord with expectations for 
a properly designed and executed random assignment process. 



347 



m>A IS-MOhrraiMPACTS/TMATMENT-CONTROLGROUPCOKIPARISON • 283 



Exhibit A. 1 Results of the Multivariate DiscriTninant Analysis of Baseline 
Characteristics of the Treatment Group and Control Group: 
The 18-Month Study Sample, by Target Group 





Adult 
women 
(1) 


Adult 
men 

(2) 


Female 
youths^ 
(S) 


Male 
youths^ 
(4) 


Incremental 


0.0032 


0.0065 


0.0128 


0.0100 


p-value for the 










incremental R ^ 


0.95 


0.29 


0.40 


0.94 


Sample size 


6,607 


5,626 


2,649 


2,144 



a, Out-of-school youths only. 
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Exhibit A. 2 Baseline Characteristics Used as Repressors in the Multivariate 
Analysis cfTreatment-Coftxrol Group Differences 

Characteristic 

EUmidty 
White, non-Hifp«nic 
Black, non-Hitpanic 
Hispanic 

Asian or Pacific Islander 
(American Indian or Alaskan Native] 

Educatioa and traininf history 
High school diploma or GED certificate 

(No high school diploma or GED certificate) 
Previously received occupational training 

(Received no occupational training previously) 
Work history 
Ever employed 

(Never employed) 
Employed upon application 

(Not employed upon application) 
Mean individual earnings in past 12 months 
Weeks worked in past 12 months 
Hourly earnings in moat recent job 
Hours worked in most recent job 

Public assistance sutus 
Receiving AFDC 

(Not receiving AFDC) 
Receiving food stamps 

(Not receiving food stamps) 
Receiving other cash assistance 

(Not receiving other cash assistance) 
Receiving housing assistance 

(Not receiving housing assistance) 

(Ccnanued) 
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Exhibii A, 2 Baseline Characteristics Used as Regressors in the Multivariate Analysis 
ofTreatment-Controi Group Differences (continued) 



Characteristic 

AFDC history 
Ever AFDC case head 

(Never AFDC caac head) 
AFDC case head 5 years or more 
AFDC case head 2 or mott, but leas than 5, years 
(AFDC case head less than 2 years) 
JTPA required for weifare, food stamps, or WIN program 
Yes 
(No) 

Household composition 
Spouse present 

(No ^ouse present) 
Own child, any age, present 

(No child present) 
Number of children under age 6 present 

Family income In past 12 months 
< $3,000 
$3,000 . $6,000 
$6,001 - $9,000 
$9,001 - $12,000 
$12,001 .$15,000 
(> $15,000) 

Living in public housing 
Yes 
(No) 

Age at random assignment 

Notes: Cbtncienstics is p4reatbe«es were left out of the rcf rcMioo equations to avoid overdeterminatioa of the 
multivaritie model. In addition to the chancteristics atentiofiod here, a constam and 19 data-availability 
indicators were used ia the eqiutions. 

a. Geoend Asnstaace or otber welfare except AFDC, food Mamps, and bousng atara pc-r 

b. WIN is the fedenl Woffc Incentive prof tmm. 



o 
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Appendix B 



A Comparison of JTPA Enrollees in the 18-Month 
Study Sample and the Title E-A Participant 
Population Nationwide 



'yms appendix compares JTPA Tide H-A enrollees in the 18-month study sample with 
S coniparison samples of adults and out-of-school youths who were 

enrolled m JTPA TiUe H-A during the sample intake period for the National JTPA Study 
(November 1, 1987 to September 30, 1989). Th, first section describes how the nationi^ 
comparison sample were constructed. The second section compares the baseline 
ntTonT"""" JTPA enrollees in the 18-month study samp? with those of the 
national comparison samples, whUe the third section compares the in-program 
experiences-duration of enrollments, program services received and JTPA perfomLce 
mdicators-for the samples. A summe y of these comparisons forms the fiiul section of 



Construction of the National Comparison Samples 

Two related comparison samples were construaed to represem the population of adults 
and out^f-school youths who entered JTPA Tide n-A programs nSonally during the 
sample int^ period for this study: one for JTPA enrollees, and one {otJfPA 
terminees. These national comparison samples were constructed from mformatio^the 
Job Traimng Quarterly Survey (JTQS), an ongoing data collection effort conducted by 
the U.S. Bureau of the Census (under contraa to the U.S. Department of Labor) and 
reponed by Westat, Inc.' ^ 



1. Sec Wcjut Inc. (1988). 
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The JTQS obtains data quarterly from the admiaistrative records of a probability 
sample comprising 142 of the 649 JTPA service delivery areas (SDAs) nationally. Two 
JTQS samples are drawn each quarter: a sample of enrollees from the 142 sites, which 
IS drawn from SDA records on persons who became enrolled m JTPA during the quarter 
(regardless of when they were terminated from the program); and a sample of terminees, 
which is drawn from SDA records on persons who were terminated from JTPA during 
the quarter (regardless of when they were enrolled). 

To construct our national comparison sample of JTPA enrollees, we obtained all the 
JTQS enroUee sample data on adults and out-of-school youths who became enrolled in 
JTPA Tifle n-A between November 1, 1987 and September 30, 1989.' JTQS sample 
members who were in-school youths or who became enrolled in JTPA before November 
1, 1987 or after September 30, 1989 were thus excluded from the sample. In the 
discussion that follows we compare the baseline characteristics of the 12,289 members 
of this national comparison sample of JTPA enrollees and the baseline characteristics of 
the JTPA enrollees in the 18-month study sample. 

Further comparisons with regard to in-program experiences are not possible, 
however, because information on the duration of enrollments, program services received, 
and program performance indicators is based on the termination date and status of JTPA 
participants, which are not available for many members of the JTQS enrollee sample who 
had not yet completed their JTPA enrollment when their data were collected. 

We therefore constructed a national comparison sample of JTPA terminees, obtaining 
all JTQS terminee sample data on those terminees who became enrolled in Title II-A 
during the sample intake period for this study (agam, from November 1, 1987 to 
September 30, 1989).' Excluded from this sample were in-school youths and persons 
who became enrolled in JTPA before November 1, 1987 or after September 30, 1989. 
Thus, although this sample is based on tenninee records, it reflects the national 
population of adults and out-of-school youths who were enrolled in JTPA Title II-A 
during the sample intake period for this study.* 



2. This infonntUon is in the JTQS PubUc Uie File produced by Wcsut. Inc. . for the period th«t bee«n 
in the second qutrtcr of program ye«r 1987 (October to December 1987) and ended in the fust quarter of 
program year 1989 (July to September 1989). 

3. This infomaUon is in the JTQS Public Use File produced by Wesut. Inc.. for the period that began 
in the second quarter of program year 1987 (October to December 1987) and ended in the fourth quarter of 
program year 1989 (April to June 1990). 

4. In other words, by choosing the national comparison wmple of JTPA terminees from within a 
represenutive naUonal sample of terminees for a period that spanned the study sample intake period and 
extended nine months beyond it. we obuined a naUonaUy n^resenUUve sample of adulu and out-of-school 
youths who became enroUed in JTPA during our intake period for whom complete dau on their enrollmenl 
penod. services received, and program performance indicators were available. 
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Uter m this appendix we compare the subsequent in-program experiences and 
performance mdicators of the 13.434 members of this naUonal comparison sample of 
JTPA tenmnees with those of JTPA enroUees in the 18-month study sample. In the next 
secuon we also present comparisons using the terminees* baseline characteristics- but 
because the terminee sample may be slightly less complete than the enrollee sample ' 
primary emphasis in the comparisons of baseline characteristics should be placed on the 
JTQS enrollee sample. Nonetheless, there was not much difference in baseline 
characterisucs between the two national comparison samples. 



A Comparison of Baseline Characteristics 

For the purposes of this comparison, we constructed measures of baseline characteristics 
that were defined in the same way as the baseline characteristics that were measured for 
tne is-month study sample using Background Information Form (BIF) responses * To 
mmimize the potential for noncomparabUity between the BIF data and the JTQS data we 
excluded any measured baseline characteristic with missing data for more than 20 percem 
of either sample. ^ 

Exhibits B. 1 through B.4 display the findings for the 16 baseline chararteristics (15 
for youths) for which comparable BIF and JTQS data were avaUable. Each exhibit 
prwents the finding for a separate target group. The baseline characteristics of em-oUees 
m the 1 8-month study sample are expressed as simple means and percentages, while those 
of the two national comparison samples are expressed as weighted means and weighted 
percentages, based on the JTQS sampling weights in the Public Use FUes.' 

Overall, the results in the four exhibits indicate that the measured baseline 
characteristics of enrollees in the 18-month study sample were quite similar to those of 
adults and out-of-school youths who became enrolled in JTPA Tide H-A programs 
nationwide during the intake period for the National JTPA Study. For none of the 
basehne characteristics measured was there a difference between enrollees in the 
18-month study sample and either of the two national comparison samples that was 
consistentiy large across all four target groups. In addition, there is no clear pattern to 
the few noticeable differences that appear within each target group 



5. U.S. Dcptrtment of Ubor (1987, 7n.). 

6. The only difference between the bmseline chmractcrisiic definition* employed in the chapten of thi* 
report .nd thoie u*ed here is th»t. unlike in the chapter,, the definition of "any public aisUunce" iwed here 
does not include unemployment insurance-receipt of which is not measured in the JTQS files. 

7. WesUt. Inc. (1989) describes the JTQS sampling design and the wcighu that reHect this design. 
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Because of the complex sampling design for the JTQS, it is not possible to calculate 
whether any baseline cbaraaeristics of the national conq>arison samples were st^istically 
significantly different from those of enrollees in the 18-month study sample/ Hence, 
many if not all of the differences teponed in the exhibits may reflect random sampling 
error rather than systematic differences among the samples. 

Turning to the specific results, we find that the 18-month study sample closely 
resembled the national comparison san^>les in terms of age. Most adults were about 
evenly split between the ages of 22 to 29 and 30 to 44. The average age of adult 
enrollees was about 33 in the 18-month study sample and about 34 in the national 
comparison samples. This slight difference is due mainly to the fact that the adults in the 
18-month study sample were less likely to be over 54 years old, because six of the study 
sites excluded older applicants from participation in the experiment. Older workers were 
not excluded from the JTQS and Aus appear with greater frequency in the national 
comparison samples. Tie youth samples were also very comparable in age, with 
enrollees having exactly the same mean age in the 18-month sample and the national 
comparison sample of JTPA enrollees. 

The ethnic compositions of the samples were also quite similar. Whites made up 
around half of both samples; blacks, from a quarter to a third; and Hispanics, from 10 
percent to 19 percent. The ethnic mix of addt men in the 18-month study sample was 
very close to that of the national comparison samples. Both adult women and female 
youths were, however, less zpt to be black— and female and male youths more apt to be 
Hispanic— if they were in the 18-month study sample than if they were in the national 
comparison sample of enrollees. 

The educational backgrounds of adult enrollees in the 18-month study sample were 
similar to those in the national comparison samples. Over two-thirds of the women and 
men in all three samples had a high school diploma or a GED certificate. But the youth 
enrollees in the 18-month study sample were less apt to have a high school credential 
than were their counterparts in the national comparison samples. 

Specifically, 47.5 percent of the female youth enrollees m the 18-month study 
sample had a high school credential versus 59.1 percent and 55.7 percent of the female 
youths in the national comparison samples. Only 38.3 percent of the male youths in the 
18-month study sample had a high school credential versus 44.7 percent and 45.5 percent 
of the male youths in the national comparison samples. Hence, it ^pears that youth 
enrollees in the 18-month study sample were more educationally disadvantaged than were 



8. Wesut, Inc, (1989) describes the likely effect of the JTQS SAinpling design on the lUtiitictl 
properties of the data for the sample. 
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their counteq)arts in the national comparison samples, although, again, it was not possible 
to determine whether this diflference was statistically significant. 

The employment statu^ of enrollees in the 18-month study sample was very similar 
to that of members of the national comparison samples. In all three samples only about 
12 percent to 16 percent of the target group members were employed when they ^plied 
to JTPA. 

The final panel in the exhibits indicates the receipt of public assistance by sample 
members. Across all four target groups the enrollees in the 18-month study sample were 
slightly less likely to be receiving any public assistance and AFDC in particular than 
were the members of the national comparison samples. In all three samples, however, 
the males (both adults and youths) were far less likely than the females to be receiving 
public assistance, especially AFDC. Thus, the patterns of receipt of public assistance 
across gender categories was the same for both samples. Furthermore, enrollees in the 
18-month study sample were about as likely as, or more likely than, the national 
comparison samples to be receivmg food stamps when they applied to JTPA. Hence, 
there was no consistently large diflference in the patterns of receipt of public assistance. 

In summary, then, it appears that the baseline characteristics of the emollees m the 
18-month study sample were generally similar to those of the national comparison 
samples. No charaaeristics differed appreciably for all four target groups and there is 
no clear pattern to the few noticeable diflFerences that exist. The main excq)tion is that 
the youth enrollees in the 18-month study sample were less likely to have a high school 
credential than were their counterparts in the national con^>arison samples. 



A Comparison of In-Program Experiences 

Exhibits B.5 through B.8 compare the duration of enrollments, the JTPA services 
received, and key performance indicators for the study sample enrollees and the national 
comparison sample of JTPA terminees. As noted earlier, this information is complete 
for only those persons with a complete JTPA enrollment record— that is, termmees; the 
exhibits therefore do not present information on the national comparison sample of 
enrollees. Complete enrollment records were available for the national comparison 
sample of JTQS terminees and for at least 18 months of follow-up for the enrollees in the 
18-month study sample.* t> 



9. In both CASCS there were some missing dau for specific items, as would be expected. 
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The data for the comparison sample of tenninees represent a single spell of 
enrollment in JTPA Title II-A per sample member.*® SDA records on study sample 
enrollees indicate, however, that 96.5 percent of this group had one enrollment spell in 
a Title II-A program, 3.5 percent had two spells, and less than one tenth of a percent had 
three spells during their 18-month follow-up period. To measure the in-program 
experiences of the study in the same way as the experiences of the comparison sample 
of JTPA tenninees were measured, we examined data on the^m Title II-A enrollment 
spell only. Hence, for both samples exhibits B.5 through B.8 present means and 
percentages for a single spell of enrollment in Title II-A for each sample member. The 
data for the national comparison sample of JTPA tenninees were weighted usmg their 
JTQS sampling weights. 

The first panel in the exhibits presents the mean duration of enrollments for the two 
samples." These results indicate that the enrollment periods for the two samples were 
quite similar, on average, for all the target groups except adult women. Adult women 
in the 18-month study sample had a somewhat shorter average enrollment period, 4.0 
months, than that of their counterparts in the national comparison sample of JTPA 
tenninees, 4.5 months." 

The second panel in the exhibits displays the percentage of sample members who 
received each of the six categories of specific JTPA services. It is in this regard that 
vcnrollees in the 18-month study sample differed the most from the national comparison 
sample of JTPA tenninees. Specifically, for all four target groups: 

• enrollees in the 18-month study sample were more likely to receive classroom 
training in occupational sldlls and job search assistance than were members of 
the national comparison sample of terminees; and 

• enrollees in the study sample were less likely to receive on-the-job training and 
miscellaneous services than were members of the national comparison sample of 
terminees. 



10. Hence, the national comptriion ttmple of JTPA tenninees is s nttionally rq>retentative sampie of 
'completed* JTPA enrollment spells that began during the sample intake period for the National JTPA Study. 

1 1 . Teimiaation dates were missing for 10.3 percent of the enrollees in the 1 8-month sttidy sample (13 .9 
percent of the adult women, 6.9 percent of the aduh men. 10.4 percent of the feniale youths, and 8.0 percent 
of the male youths). Because none of the recorded enrollment periods was longer than 12 months, the fact 
that termination dates were not recorded for these sample members during their 18-month follow-up period 
probably represenu missing data rather than enrollment periods that were longer than 18 months. 

12. Mean enrollment periods for the 18-month study sample in exhibiu B.5 through B.8 are longer than 
the corresponding median enrollment periods shown in Exhibit 3.20 in Chapter 3, as would be expected. 
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In addition, for all the target groups except adult women, enrollees in the study sample 
were more likely to receive basic education than were members of the comparison sample 
of terminees, although the differences for this fifth category of services were smaller than 
those for the other four. 

It is unlikely that all of these differences in service receipt are due to differences in 
how the data for the two samples were collected or how specific JTPA services were 
defined. Measures of service receipt for both samples were construaed from SDA 
enrollment and tracking data on individual sample members. And the definitions of each 
specific program service from each data source were carefully compared." 

On the other hand, because it is not possible to determine whether the differences 
were statistically significant, it is difficult to know whether they represent the result of 
random error due to sampling or real, systematic differences between the study sample 
and its counterpart of adults and out-of-school youths in Title II-A programs nationally. 
Nevertheless, given the magnitude and consistency of the observed differences in the mix 
of services received by the two samples, the data suggest but cannot prove that these 
samples represent groups that received somewhat different services from JTPA. 

Despite these potential differences, however, the second panel also suggests that the 
two samples were similar in the patterns of service receipt across target groups. Perhaps 
most striking in this regard is the much greater likelihood of receiving basic education 
among youths than among adults in the two samples. Second is the much greater 
likelihood of receiving classroom training in occupational skills among females (both 
adults and youths) than among males. And third is the greater likelihood of receiving on- 
the-job training among males than among females. 

The last panel in exhibits B.5 through B.8 indicates very little difference in key 
indicators of program performance between the 18-month study sample and the national 
comparison sample." First, for all the target groups except female youths the entered 
employment rate (the percentage of sample members who had found a job before 



13. Published JTQS rq>on$ use an algorithm to classify each sample member according to the primary 
service (referred to as a ^'program activity") that he or she received, and thus include each sample member 
in one service category only. Here, to match the way that service receipt rates are measured throughout this 
report, we include each JTQS sample member in every service he or she received, and thus, each sample 
member can be represented in more than one service category. The findings in exhibits B.5 through B.8 
are therefore not directly comparable to those in the published JTQS reports. 

14. Of the 6,559 enrollees in the Ig -month study sample who had a valid termination date, 24 were 
missing dau on whether or not they were employed upon their termirution from JTPA. Of the 4,200 
enrollees in the 18-month study sample who had a valid termination data and who were employed upon 
termination, 420 were missing data on the wage rate for their job. 
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terminating their enrollment in JTPA) was very similar for members of both samples; for 
female youths the entered employment rate was 51.1 percent for enrollees in the 18- 
month study sample and 57, 1 percent for the national comparison sample of terminees. 
Second, for all except adult women the average hourly wage for sample members who 
were employed when they were termmated from Title n-A was very similar for the two 
samples. For adult women this "placement wage" waj $5.06 for members of the study 
sample and $5.99 for members of the national comparison sample. 

Hence, there is no evidence that the study sample examined in this vepon 
experienced a level of program performance that differed ^preciably from that 
experienced by members of the national comparison sample of JTPA terminees." 



Summary 

Although the 18-month study sample was not drawn from a national probability sample 
of SDAs, it resembled its counterpart of adults and out-of-school youths in the Title II-A 
program nationally. Nevertheless, it also differed from the two national comparison 
samples in several important ways. 

In terms of baseline characteristics the 18-month study san^le is quite similar to its 
national counterpart. The only ^preciable difference is that out-of-school youth 
enrollees in the 18-month study sample were less likely than out-of-school youths in Title 
II-A programs nationally to have a high school diploma or GED certificate when they 
applied to the program. 

In terms of their experiences in JTPA programs the enrollees in the 18-month study 
sample bad about the same average duration of enrollments as their counterparts 
nationally, they were about equally likely to be employed upon their termination from the 
program, and they earned about the same average hourly wage if they were employed. 

In terms of JTPA services received, however, the enrollees in the 18-month study 
sample were more likely to receive classroom training in occupational skills and job 
search assistance, and less likely to receive on-the-job training and miscellaneous 
services, than were members of the national comparison sample of JTPA terminees. 
Nevertheless, the patterns of differences in service mixes across target groups were quite 



15. A third key indicator used to measure JTPA peifonnancc for youths is the ''positive iermination 
rate,** which i$ the percenuge of all youth terminees who, before terminattng their JTPA enrollment, had 
found a job , atuined recognized employment competencies esublishcd by the Private Industry Council (PIC), 
completed elemenury, secondary, or post-secondary school, enrolled in another training program or an 
apprenticeship, enlisted in the Armed Forces, or returned to school full-time. Because dau on positive 
termination rates were missing for more than 20 percent of the youth enrollees in the IS-month study sample, 
we did not compare findings on this indicator. 
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similar for the two samples. In both the study sample and the national comparison 
sample of terminees, the youths were more lUcely dian the adults to receive basic 
education, the females were more likely th . the males to receive classroom training in 
occupational skills, and the males were more likely than the females to receive on-the-job 
training. In this regard the experiences of the 18-month study sample reflect decisions 
by local SDA staff, entry requirements and service availability of local programs, and 
personal preferences of the applicants themselves that appear to be similar to those which 
determined the mix of services received by Title n-A participants across the country. 

But whatever the differences or similarities we may observe between the 18-month 
study sample and the two national comparison samples, there is no way to determine how 
the program impacts produced at the 16 study sites compare to those produced by JTPA 
Title n-A programs nationally, because there is no valid measure of average program 
impacts nationally. 

Hence, we make no claims about the national representativeness of the impact 
estimates presented in this or any other report for the National JTPA Study. Instead, we 
present our findings as representing a broad range of different SDAs, which served many 
different types of participants, under widely varying economic conditions, within the 
context of local institutional arrangements that also varied substantially. 
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Exhibit B. 1 Selected Baseline Characteristics of Adult Women: JTPA EnroUees . 

in the 18-Month Study Sample and the National Comparison Samples 
of JTPA EnroUees and Temunees 



Characteristic 



18-month 
study sample 

JTPA enrolUes 

01 



National comparison samples 



JTPA enroUees 



JTPA terminees 
m 



Age 

< 22 
22-29 
30-44 
45-54 
> 54 
Mean age 
Ethnicity 

White» non-Hispamc 
Black, non-Hispanic 
Hispanic 
Other^ 

Education status 

HS diploma or G£D certificate 
No HS/GED 

Employment status 

Employed 
Not employed 

Public assistance status 

Receiving any public assistance ^ 
Receiving AFDC 
Receiving food stamps 
Sample size 



44.0% 
44.0 

7.4 

4.6 

33.2 

56.2% 
27.9 
12.2 
3.6 

73.8% 
26.2 

15.0% 
S5.0 

54.4% 
34.6 
49.6 
2,883 



42.6% 
41.5 

8.1 

7.9 
33.8 

54.0% 
32.0 
11.0 
3rl 

70.9% 
29.1 

15.2% 
84.8 

60.2% 
40.4 
47.8 
5,032 



41.8% 
42.1 

7.6 

8.5 
34.4 

53.9% 
32.3 
10.5 
3J 

71.7% 
28.3 

14.9% 
85.1 

61.4% 
36.3 
50.1 
5,395 



Sources: UoidjuAed fre4]ueacie< bued on B^cktrouad IttfomattoQ Form reipofiiet tad weif fated finequeacies 
based od Job Tfiiwaf Quaiteriy Survey (iTQS) d«a oa JT?A earaUeea aad tenniaeet wbo becatte carolled 
in rule n-A betwDca Novenber 1, 1987 tad Seple^wr 30, 19t9. 

a. At rudom atngniaeat (18*moath «udy Mople) or opoa tppUcetioa (aatioaal coflaparitoo — ipk^) 

b. This caleg ory tachidet Aaehcaa ladiaat, AlaifcaB Nativca, Aaiaas, aad PaciSc Iilaaden. 

c. *Any public aiaMace* iacludet the foUowiog aourcea of aaatfaaoe: AFDC, food Munpt, bouaac aMiMance, 
aad otber cash aniMaace. h does aoc iaclude vaemployaieiK iaairaace. 
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ExhibixB.2 Seleaed Baseline Characteristics of Adub Men: JTPA Enrollees 

in the 18-Month Study Sample and the National Comparison Samples 
of JTPA Enrollees and Terminees 





IS-month 








study sample 


National comparison samples 




JTPA enrollees 






Characteristic 


(1) 


(2) 


(3) 


Age^ 








< 22 


- 


— 




22-29 


45.5% 


43,2% 


43.4% 


30 - 44 


42,3 


42.2 


40.9 


45-54 


7.8 


7.4 


7,7 


> 54 


4,5 


7,2 


7.9 


Mean age 


33,0 


34,2 


34,2 


Ethakitj 








White, non-Hispanic 


57,6% 


58.4% 


58.0% 


Black, non-Hispanic 


27,6 


26.2 


27.1 


Hispanic 


10.3 


11.8 


11.4 


Other ^ 


4.5 


3.6 


3.5 


Education status 








High school graduate or GEO 


70.3% 


69.6% 


67.8% 


Neither 


29.7 


30.4 


32.2 


Employment status 








Employed 


12.7% 


14.3% 


13.6% 


Not employed 


87.3 


85.7 


86.4 


Public assistance 








Receiving any public assistance^ 


30.4% 


34.9% 


34.8% 


Receiving AFDC 


6.6 


10.1 


8.9 


Receiving food stamps 


2S.5 


25.8 


26.5 



Sample size 2,286 3,835 4,293 

Source*: Ufiadjuitod frequcnciec h^tod oo B«ckf rouad lafonMiioa Pom mponiea tad weif htad fr«que«ciec 
bMod oa JTQS <Uu oo JTPA carolled aad tenniDoea who bec«ae cwotkd ia Tide D-A betweea Novcnfaer 1, 19t7 
AniSqxeiBber30. 1989. 

a. At nadom aMi{ome«t (iS-oio^ fludy Mnple) or «poa applieotioa (MCiooal cooiparitM tmapiu), 

b. This catef ory iacludes Aaiericaa ladiiai, AWikaa NMivet, AaMt, aad Pacific liUadcn. 

c. *Aoy public aiatfaacc* iackidet the foUowiac source* of uaifUaoe: AFDC. food tfao^ kouaag aMtiCaace, 
and ocber caih aiviiuace. h doe« aot tacHide Macfbyfrt iawmacc. 
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Exhibit B.3 Selected Basetine Characteristics of (ha-of -School Female Youths: 
JTPA EnroUees in the 18-Month Study Sample and Ae National 
Comparison Samples of JTPA EnroUees and Terminees 





18-month 










National comparison samples 




JTPA enroUees 


JTPA enroUees 


JTPA terminees 


Characteristic 


(1) 


(2) 


(3) 


Age- 








16- 19 


59.3% 


61.0% 


60.4% 


20-21 


40.7 


39.0 


39.6 


> 21 




- 




Mean age 


19.0 


19.0 


19.0 


Ethnicity 








White, non-Hispanic 


51.2% 


48.0% 


48.2% 


Black, non-Hispanic 


28.2 


37.2 


34.2 


Hispanic 


18.8 


11.7 


14.9 




1.8 


3,2 


2.8 


Education status 








HS diploma or GED certificate 


47,5% 


59.1% 


55.7% 


No HS/GED 


52.5 


40.9 


44.3 


Employment status 








Employed 


15.6% 


13.3% 


12.1% 


Not employed 


84.4 


86.7 


87.9 


Public assistance 








Receiving any public assistance ^ 


45.9% 


53.0% 


54.0% 


Receiving AFDC 


26.6 


27.1 


30.3 


Receiving food stamps 


39.6 


39.5 


42.1 


Sample size 


1,188 


1,725 


1,920 



Sources: Uni<$u«ed frequeacies bited on B*cktrouDd In f omal i oo Fonn re^KNiaes and weichted frequencies 
buedofinX2SdaUOQnTAewoUee«aadiefBtBe«»wbobec«aeearoltodtaTiUen-Abet^^ 1. I9S7 



aodScpteniberaO. 1989. 

a. At nadom mpuneax (IS-ncoth study Miople) or upoo appUcMioa (oatiooAl comptrisoii samples). 

b. This category tachides Ajnericafi bdians, Alajiuui Natives, Aaaas, aad Pacific IsUaden. 

c *Aay public asaistaoce* iacludes tbe fdlowiaf sources of a«istaace: AFDC, food sunps, hoaunt asaistaace, 
aod other cash assi«aoce. h does oM iadude upeoplgyacat iasufiace. 
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Exhibit B.4 Selected Baseline Characteristics of Om-of-School Male Youths: 
JTPA EnroUees in the 18-MonA &udy Sample and the National 
Comparison Samples of JTPA EnroUees and Terminees 



Characteristic 



IS-manih 
study sample 

JTPA enroliees 
(I) 



National comparison samples 



JTPA enroliees 

(2) 



JTPA terminees 



Age 

i6- 19 
20-21 
> 21 
Mean age 
Ethnicity 

White» non-Hi«panic 
Black» non*Hisp«nic 
Hispanic 
Othcr^ 

Education status 

HS diploma or GED certificate 
No HS/GED 

Employment status 

Employed 

Not employed 
Public assistance status 

Receiving any public ascittance ^ 

Receiving AFDC 

Receiving food stamps 
Sample size 



63.9% 
36.1 

18.9 

55.2% 
26.5 
15.8 
2.4 

38.3% 
61.7 

12.1% 
87.9 

29.2% 
5.7 
26.0 
959 



61.7% 
38.3 

18.9 

55.7% 
28.8 
12.2 
3.3 

44.7% 
55.3 

13.7% 
86.3 

32.8% 
8.3 
23.4 
1,697 



63.2% 
36.8 

18.8 

54.2% 
27.7 
15.8 
2.3 

45.5% 
54.5 

13.2% 
86.8 

32.1% 
9.0 
25.2 
1,826 



Source: UiMd jutted fnqueacic* bued oft B«ckfrouad Ufonottioa Bonn rc«poiMe« ukI wdsbtcd fttiqueacic* 
bMcd on XT<^dAU on HTAcaroUeet aid iemiMi«4o became e«^^ 19S7 
AttS Scfxembcr 30. 19S9. 

a Al rmndott aaaif nmc^ ni^moaA Mitdy mmfiS^) ar upm fpUgM^ft* ^^wtyftfWM — ^flm), 

b. Thif c«icf ory iadiadec Aaericw ladiau. AMcm NMivot, Atkat, cad Pacific UaMien. 

c. 'Aay public aiaMaace* iacliadet tlie foUowiaf tourcet of ■■■tfi>-r- APDC» food Man^ koMtii^ auiaLatt:e, 
aod other catb ait i iup cc . h doei not tacludc yaenployBeat iMuraacc. 
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Exhibit B.5 In-Program Experiences of Adult Women: JTPA Enrollees in the 
IS-Month Study Sample and the National Comparison Sample of 
JTPA Terminees 





1fl-ntr%nth 


N/sti/yn/il 




study sample 


comparison sample 




JTPA enroUees 


JTPA terminees 


In^progr^im txp€Ticnc€ 






Mean number of months 








4.0 


4.5 


Percentage who received: 






Classroom training in 






occupational skills 


47.296 


35.2% 


Basic education 


12.3 


12.8 


On->the<job training 


19.5 


25.0 


Job*search assistance 


33.6 


25.8 


Work experience 


4.9 


4.0 


c 

Miscellaneous 


21.1 


30.4 


Performance indicator 






Entered employment rate 


64.8% 


66.2% 


Mean hourly wage» if 






employed at termination 


$5.06 


$5.99 


Sample size 


2,S&3 


5,395 



Sourcet: Uo.idjtuKed auoHmeot «a4 tnddag 6aiM, from the 16 SDAi 

ftod weighted JTQS dau oo JTPA tennineec who became enrolled in ITtle D-A betweea 

November 1, 19S7 aod September 30, 19S9. 

a. During the first forma] ipdl of Title D-A eorollmcnt duriog the IS-mooth follow-up period. 

b, "Basic education* iocludea Adult Batic Educaftkm (ABE), high school or General Educational 



Deveiopmenc (GED) preparation, and English aa a Second Language (ESL). 

c. *Miicdlaneous* includes assessment, job-readiness training, cuaomized training, 
vocational exploration, job diadowing, and ttyout employment, among other services. 

d. The "entered employment rate* is the percentage of all tenninaes who had found a 
job before terminating their enrollment in JTPA. 
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Exhibii B. 6 In-Program Experiences of Adult Men: JTPA EnroUees in the 
18-Month Study Sample and the National Comparison Sample of 
JTPA Terminees 





IS-month 


National 










JTPA enrollees 


JTPA terminees 


In-program experience 


(1) 


a) 


Mean aumber of months 


3.3 


3.5 


enrolled 






Percentage who received: ^ 






Classroom training in 






occupational skills 


26.3% 


19.9% 


Basic education^ 


9.1 • 


7.2 


On-the-job training 


25.5 


36.3 


Job-search assistance 


40.1 


28.6 


Work experience 


3.0 


4.0 


Miscellaneous ^ 


21.8 


30.6 


Performance indicator 






Entered employment rate 


72.2% 


72.2% 


Mean hourly wage, if 






employed at termination 


$5.78 


$5.71 


Sample size ^ 


2,2W 


4,293 



Sources: Unjuijusted enrollment and tracking dau from the 16 SDAt and weighted JTQS 
data on JTPA termineea who became enrolled in Title II-A between November 1 , 1987 
and September 30» 1989. 

a. During Ihe fint formal spell of Title D-A enrollment during the 18-month follow-up period. 

b. "Basic education" includes Adult Buic Education (ABE), high school or General Educational 



Development (GED) preparation, and English as a Second Language (ESL). 

c. "Miscellaneous" includes assessment, job-readineu training, customized training, 
vocational e^qploration, job shadowing, and tryout enqsloyment, among other services. 

d. The "entered employment imte" is the peroentage of all terminees who had found a 
job before terminating their enrollment in JTPA. 
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Exhibit B. 7 In-Program Experiences of Out-of-School Female Youths: JTPA 

Enrollees in the 18-Month Study Sample and the National Comparison 
Sample of JTPA Terminees 





18'fnonth 


National 




study sample 


comparison sample 




JTPA enrollees 


JTPA terminees 


In-program experience 


(1) 


a) 


Mean number of months 


4.4 


4.4 


enrolled 






Percentage who received: 






Classroom tnising in 






occi^>atioDal skills 


43.1% 




Basic education 


27.9 


21.6 


On-the-job training 


14.3 


20.9 


Job-search assistance 


34.5 


20.3 


Woric experience 


7.4 


9.5 


Miscellaneous 


21.9 


32.2 


Performance indicator 






Entered employment rate 


51.1% 


57.1% 


Mean hourly wage» if 






employed at termination 


$4.43 


$4.53 


Bangle size 


1J88 


1,920 



Sounec: Unadjuited enrollment and tncking daU from the 16 SDAs and weighted JTQS daU 
on JTPA tcrminoct who became enrolled in Title 0-A between November I, 19S7 and 
September 30, 1989. 

a. During the fmt fomml tpell of TiWt fl-A enrollment during the 18-month foUow^p period. 

b. "Basic education* includee Adult Banc Education (ABE), high school or Genenl Educational 
Development (GED) prepantion. and English as a Second Language (ESL). 

c. "Misocllaneoui" includes assessment, job-readiness truning, cuAomizod txaining, 
vocational exploiition, job shadowing, and tiyout en^loyment, among other services. 

d. The "enured employment rate" is the percentage of sU terminees who hsd found a 
job before temiinsting their enrollment in JTPA. 
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ExhibU B.8 Jn-Frogram Experiences of Out^fSchool Male Yowfhs: JTPA 

EnroUees in the IS-Momh Study Sample and the Naxional Comparison 
Sample of JTPA Terminees 



JS-month 
study sample 

JTFA trtrollees 



National 
comparison sample 



JTPA terminees 



In-program experience 


(1) 


(2) 


Mean Dumber of months 






enrolled 




3.8 


PercenUge who received: ^ 






Classroom training in 






occupational skills 


28.1% 


19.7% 


Basic education^ 


26.5 


19.8 


On-the-job training 


19.2 


27.8 


Job-search assistance 


36.3 


19.8 


Woiic experience 


6.8 


8.8 


Miscellaneous ^ 


26.6 


36.5 


Performance indicafi^r 






Entered employment rate 


59.5% 


60.9% 


Mean hourly wage, if 






employed at termination 


$4.77 


S4.83 


Sample size 


959 


1826 



Sources: Uudjusied eoroUmeat «Ad tivckioa dau from the 16 SDAs and wctsbted 

JTQS d«4 OA JTPA icnniooes who became enrolled in Title H-A betweea November 1 . 1987 

•Ad September 30, 1989. 

a. Dunof the first fonna] tpell of TiUe II-A ewollmeik dunna tbe 18-momh follow-up period. 
'Basic education* includes Adult Basic Education (ABE), bi^h scbool or General Educational 
Development (GED) preparation, and Enalish as a Secoad Lanaua^e (ESL). 
'Miscellaneous* includes assessment, job-readiness trminia^, cu«onized trviaing, 
vocational exploration, job shadowing, and tryout employment, amonf ocber services. 
Tbe 'entered employment me* is the perceatag e of a2£ terminees who had found a 
job before tenninatina their enrollmefit in JTPA. 
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Appendix C 
Data Sources for the 18-Month Impact Analysis 



ERIC 



THE impact analysis presented in this rq)ort is based on the experience of the 17,026 
members of the experimental sample who were scheduled for a First Follow-up 
Survey interview 18 or more months after their random assignment— the 18-momh study 
sample} This appendix describes five of the six data sources we used in analyzing the 
impacts on this sample: 

• the random assignment telephone file compiled duting the intake of the 
experimental sample and covering all 20,601 experimental sample members, 

• Background Information Form responses, collected for 20,501 experimental 
sample members upon their application to JTPA, 

• JTPA enrollment and tracking data provided by the 16 service delivery 
areas (SDAs) that served as study sites, 

• First Follow-up Survey responses, collected from interviews with 17,217 
members of the experimental sample, 

• earnings data from state unemployment insurance (Ui) agencies for members 
of the 18-month study sample in 14 of the 16 study sites, and 

• data on site characteristics collected as part of a study of the implementation 
of the experiment. 



1. Appendix D describes our definition of the iludy sample in more deUtl. 
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The last source, data on site charaaeristics, is described in Doolittle (1992), a companion 
report on the design and implemenution of the National JTPA study. 

The following sections describe each of the first five data sources in turn, reviewing 
the content of and method of collecting the data, the completeness of the data, and the 
construction of analysis variables from the data source. Detailed documentation of each 
source and its derivation, as well as copies of the Background Information Fonn and the 
First Follow-up Sur\'ey instrument, are available from Stephen Bell at the Bethesda 
offices of Abt Associates. 



The Random Assignment Telephone File 

The random assignment telephone file was compiled at the time JTPA applicants at the 
16 study sites were randomly assigned to treatment group or control group status. 
Specifically, after an SDA sufF member had determined a program 25)plicanfs eligibility 
for the program and assessed and recommended him or her for one of the three service 
strategies, the staff member would call The Manpower Demonstration Research 
Corporation (MDRC), where a computer program would randomly assign the applicant 
to treatment group or control group status (See Exhibit 2-1 in Ch25)ter 2), As part of this 
process the computer program generated a file of identifiers and basic descriptors for the 
20,601 members of the ftiU experimental sample. 



Content and CoLLEcnoN Method 

These identifiers and descriptors all came from information in the Background 
Information Form (BIF) that the applicant had completed at application. The SDA sufF 
member read the following BIF data over the telephone: the sample member's name. 
Social Security number, date of birth, gender, ethnicity, reconmiended services and 
service strategy, and service delivery area (SDA). The random assignment computer 
program then computed four additional variables: age at rac'^om assignment (from the 
current date and date of birth), target group (from age, gender, and ethnicity), datr of 
random assignment (the computer's internal date), and treatment or control group status 
(also generated within the computer, at random, using a 2/1, 3/1, or 6/1 treatment- 
control group ratio, as explained in Chapter 2). On a regular basis throughout the period 
of random assignment, new entries in the file were transmitted electronically to the study 
data base maintained at Abt Associates. 
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Completeness 

Because all of the identifiers and descriptors for a given program applicant bad to be 
(K)mplete for random assignment to take place, the random assignment telephone file 
contains no missing data. 



CONCTRUCnON OF ANALYSIS VAIOABLES 

The random assignment telephone file was merged with the BIF file at Abt Associates 
before either file was edited or used to construct analysis variables. The telq>hone file 
thus served as a check on the accuracy of Ae information taken from the BIF (see below) 
and as a source for some of the variables used in construaing the impact analysis file. 
When in conflia, values from the BIF were accq)ted over values from the telephone file 
for all variables except those generated internally to the computer during the random 
assignment call, namely, the date of random assignment, and treatment-control group 
status. 



The Background Information Form 

Background Information Form responses serve as the basis for the vast majority of the 
baseline characteristic variables used in the impact analysis.^ These variables are used 
for several purposes: to describe the characteristics of the sample; to define the target 
gioups, service strategy subgroups, and other key subgroups examined in the analysis; 
and, as covariates in the impact regressions, to control for differences in baseline 
characteristics between the treatment and control groups. 

Content and Collection Method 

The BIF provides information on sample members* demographic and household 
characteristics; earnings, income, and income sources, including public assistance; work 
education, and training histories, and other characteristics as of the time the sample 
members applied, to JTPA. A copy of the four-page form, whidi details the specific 



2. As noted above, two exceptions tre the vtritbles for the date of nndom SAsignment and treatinent- 
control sutus, wh^b generated from the nmdom assignment telephone file. A third exception is 
earnings data from state UI agencies, described in the last section of this appendix. 
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variables collected in each of these categories, appears in Bloom (1991), an earlier report 
in this series, and is also available from Stephen Bell at Abt Associates in Bethesda. 

Most items on the BIF were filled out by sample members during the JTPA intake 
process, with help from SDA staff as needed. Three key variables were recorded directly 
by SDA staff, however: the SDA, the specific program, services recommended for the 
sample member, and the service strategy encompassing those recommended services. 

The BBF also served as the SDA*s record of the random assignment telq)hone call.^ 
As noted above, all of the variables reported to MDRC during the call came from the 
BIF. The SDA staff entered the treatment-control group status onto the form during the 
call and then made a copy for SDA records. Con^)leted forms were mailed to Abt 
Associates for data entry and double-key-entry verification. 



Completeness 

BIFs were collected for almost all experimental sample members: 20,501 of 20,601 
individuals, or 99.5 percent. For 55 of the 64 variables on the form, usable data were 
obtained for 90 percent or more of the sample. All but 2 of the 64 variables (unem- 
ployed/not in the labor force and months since left school) had response rates for all 
target groups of over 80 percent.* 



Construction of Analysis Variables 

Once entered into the Abt Associates data base, all the BIF variables were checked for 
out-of-range values or violations of the form's skip patterns (that is, answers to questions 
that should have been skipped or skips of questions that should have been answered). 
Unallowed values were coded to missing, and skip pattern violations were resolved either 
by inferring the correa answer to a question from other related responses or coding all 
conflicting variables to missing. 

The edited dau were then converted into analyst" format through a series of variable 
construction steps. Most of these transformations involved the recording of categorical 
responses (such as ethnicity) into a set of dummy variables set equal to 0 or 1. More 



3. The fonn tlso explained the confidentitlity of the dtu tnd lolicited the tpplictnt'i (or the tpplicwil's 
parent's or guardian's) pcnnission to secure information on the applicant from other public agencies. 

4. For more dcuil see Bloom (1991, Appendix C). 
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cx)inplex recodings converted whole sets of variables into the analytic concept they were 
intended to measure (such as the transformation of pay period, hours worked per week, 
and pay per pay period into the hourly wage on the most recent job). These converted 
variables were coded to missing when any of their source variables was missing. 



Enrollment and Tracking Data from the 16 SDAs 

For standard reporting purposes JTPA service delivery areas maintain machine-readable 
records on all individuals enrolled in JTPA under Title n-A. These data are used in this 
report to identify the following variables: treatment group members who became 
enrolled in JTPA (JTPA enrollees); control group members who became enrolled in 
JTPA, despite the experiment's embargo on their participation (crossovers); the JTPA 
services received by both enrollees and crossovers; and the date when the enrollees were 
formally "terminated** from the program. 



CoNTE^f^ AND Collection Method 

Data from the SDAs* management information systems show enrollment and termination 
dates for spells of JTPA enrollment, as well as the start and stop dates of each specific 
program service received during an enrollment spdl. Multiple— and sometimes 
overlapping— services durpg an enrollment spell were common, whereas multiple spells 
of enrollment occurred on occasion but never, of course, overlapped. 

Each of the SDAs in the study provided MDRC with a comprehensive, machine- 
readable file of all Title II-A enrollment spells that began during the sample intake 
period— November 1, 1987 to September 30, 1989— and expended as far as November 
30, 1990, in most SDAs. MDRC stafi" then extracted extended enrollment information 
on the 20,601 experimental sample members, using the Social Security numbers that 
appeared on the BIFs; matches of the Social Security numbers with individuals in the 
SDA files were verified using the name and date of birth, where available.^ 



5. In the CMC of the Northcait (Ft. Wayne) Indiana tite. SDA fUff rather than MDRC suff extracted 
the dau on aample memben and then tent it to MDRC. 
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Completeness 

By definition, the SDA data contain complete records of the formal enrollment in JTPA 
of all sample members in the program years covered. Hence, there are no missing 
records in the program participation files. As explained below, however, problems with 
spell dates occasionally resulted in missing data on specific dates in the file. 



Construction of Analysis Variables 

The first step in constructing the analysis variables was to ^ply a series of date edits to 
any missing or inconsistent dates in the source files (such as dates of service receipt that 
fell outside of an enrollment spell or stop dates that preceded st^rt dates). A small 
fraction of all dates emerged from this process coded as missing; dau on the spells 
involved were excluded from the analysis.^ 

The data on enrollment spells w^e then converted into a series of dummy variables 
set equal to 0 or 1 for each of the six categories of specific program services: classroom 
training in occupational skills, basic education, on-the-job training, job search assistance, 
work experience, and miscellaneous slices. Each set of variables indicates whether the 
sample member was enrolled in a Title n-A service in that category during the 18 months 
after his or her random assignment, with a s^arate dummy variable for each month. 
The algorithm used to convert the SDA service codes into the six service categories 
differed by site and is available from Fred Doolittle at the Mai^wer Demonstration 
Research Corporation in New York. 



The First Follow-up Survey 

The outcome and impact estimates in this report are based on responses t^ a survey of 
experimental sample members, conducted between November 1989 and December 1990. 
The First Follow-up Survey attempted to interview the 20,501 experimental sample 
members for whom BIF contact data were available and succeeded in interviewing 
17,217. Responses to a second follow-up survey, conducted between July and December 
1991, will be examined in our forthcoming final report. 



6, A more complete dctcription of thcic dau preptrmtion tciivitici tppean in Kemplc. DoolittJc, tnd 
Wallace, (forthcoming). 
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Content and Collection Method 

First Follow-up Survey responses offer a continuous history of sample members' 
employment and related activities over the first 13 to 39 months after their random 
assignment. More specifically, the survey queried respondents about spells of 
employment, school, and training. Selected characteristics of jobs held— including the 
number of hours worked, wage rate, overtime and bonus pay, and leave without 
pay— were collected for each job, as were the type and number of hours spent in each 
spell of school or training. Copies of tiie survey instrument, which provides details on 
the roughly 1,200 variables collected during cadi interview, are available from Stephen 
Bell at Abt Associates in Bethesda. 

Each follow-up interview was carried out by telephone if possible or in person if 
not. Interviewers from the National Opinion Research Center (NORC) in Chicago used 
contact data from the BIF, as well as address diecks through credit bureaus and other 
sources, to locate respondents. Names and telqphone numbers of friends and relatives 
(from the BIF) were also used for conuct purposes. Interpreters helped to conduct 
interviews with a small number of respondents who did not speak English. 

Computer-assisted telephone interviewing, or CATl, was used for all telephone 
interviews. In this system, a computer program displays each question on a computer 
screen for the interviewer to read, and then records each answer as it is given and 
entered by the interviewer. CAT! provides tight control over skip patterns and prompts 
the interviewer for corrections when out-of-range values are entered. On-site 
interviewers conducted in«person interviews in respondents* homes or other convenient 
locations. Responses to these interviews were first recorded on p^er and then keyed 
into CATI in the NORC central office. 



Completeness 

The survey attempted to interview all 20,501 experimental sample members for whom 
contaa data were available.^ Completed mterview records were obtained for 17,217 
individuals, or 84 percent of the sample. This total breaks down into 9,368 telq)hone 
interviews (46 percent) and 7,849 in-person interviews (38 percent). More detailed 
information of survey response rates appears in Chapter 2 (see Exhibit 2.5); a discussion 
of the implications of nonresponse for the impact analysis appears in Appendix D. 



7. One hundred stmplc mcmbOT had no EIF. and hence no conuct <Uu with which to initiate an 

interview. 
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Of the variables used to compute the central employment and earnings outcomes for 
this report, item-specific response rates met or exceeded 90 percent in all cases. 



Construction of Analysis Variables 

Several of the initial stq)S in survey variable construction paralleled those described 
above for the BIF: linkage to the telephone file, editing of out-of-range values and skip 
patterns, and construction of 0/1 dummy variables from selected categorical responses. 
A much more extensive variable construction process was used to convert dau on 
employment and school and training spells into a set of monthly activity measures. 

The first step in this process produced a set of variables indicating the share of each 
month spent in employment, job search, and school or training. If any dates for a spell 
were incomplete or inconsistent with other spell dates, all the analysis vyjiables pertaining 
to that activity were coded to missing in all ponths. Monthly variable for earnings aiul 
employment were given the missing code for 0.9 percent of the respondents; monthly 
variables for school and training were given the code for 0.7 percent of the respondents. 

Where the special missing code was not used, summary measures of monthly 
activities were constructed Oiours worked, earnings, and hours of school and training) 
and extreme values deleted.' Missing or deleted values then were replaced with imputed 
values as described in Exhibit C.l. Where more than one spell of employment or school 
or training occurred within a single month, the monthly activity measures sum across 
spells.^ 

Selected monthly aaivity measures were then summed across months to produce 
quarterly and 18-month totals for: 

• total earnings exclusive of odd-job earnings; 

• total hours worked, including overtime hours but excluding odd-job hours 
(which were not measured); 



8 . Monthly valuct exceeding 347 houri of work and 250 houri of tchooling were deleted, affecting 0.6 
and 0.02 percent of all per»on-montht , retpeclively . Earnings dau were deleted in the 0.7 percent of peraon- 
months in which the ratio of earnings to houn worked (earnings per hour) fell below $.50 or exceeded 
$50.00 and in the person-months in which hours exceeded 347. 

9. Further deuii> on Uie monthly variable construction process arc available from Stephen Bell of Abl 
Associates. 
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• proportion of the time period employed, converted into weeks worked by 
multiplying by the number of weeks in the period; 

• a 0/1 indicator of employment in the period, determined by whether hours 
worked (exclusive of odd jobs) equaled or exceeded zero in the period; 

• total hours of classroom training in occupational skills and, separately, basic 
education; and 

• 0/1 indicators of the receipt of any classroom training in occupational skills 
and, separately, basic education, during the period, determined in each case 
by whether the corresponding hours measure equaled or exceeded zero for 
the period. 

Where the survey data covered only a portion of tiie eighteenth month after random 
assignment, reported earnings, hours worked, and hours of school or training for that 
month were divided by the share of the month covered by die survey data before 
including them in any of these aggregate measures. 

Finally, additional checks of the data were conducted to ensure that unusually large 
monthly and 18-month earnings values did not result from errors in the survey data. For 
each target group the top 1 percent of the 18-month earnings distribution was examined 
to determine whether any individual earnings amounts were in error." So too were 
any individual monthly earnings amounts in excess of $5,000. These checks resulted in 
the discovery of 84 erroneous 18-month earnings totals among the roughly ISO cases 
examined. Checks of cases further down the earnings distribution were i^t conduaed 
due to their high cost^^ and low likelihood of detectmg errors large enough, and 
concentrated enough in either the treatment or the control group, to noticeably affect 
estimates of program impaa.'* Where errors were found, we rq)laced each monthly 



10. Classroom training in occupational skills includes all training received from 2- and 4>year colleges, 
graduate/professional schools, and vocational schools. Basic education includes high school, GED programs, 
adult basic education, and English as a Second Language (ESL) and other special Ittermcy programs. 

1 1 . This resulted in checks of earnings amounu down to $26,000 for aduk women, $36,000 for adult 
men, $19,000 for female youths, and $24,000 for male youths on an annualized basis. 

12. Errors were identified through a deuiled review of the sclf-repoited job descriptors on which the 
earnings measures were based. Most errors resulted from misrcportcd overtime or bonus earnings. 

13 . Each check required a deuiled examination of the timing and chancteristics of all jobs reported on 
the survey and case-by-case determinations of the reliability of dozens of individual variables in telatton to 
one another. 

14. We tested the sensitivity of the impact estimates to the deletion of additional earnings values and 

found them to be robust. 
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earnings measure with its mean for nonerroneous data on "workers** in the same target 
group, service strategy subgroup, and tre -ttment or control group*" 

A final analysis variable taken from the survey indicates whether a sample member 
atta'med a training-related high school diploma or General Education Development 
certificate (GED) at any point during the follow-up period, including points more than 
18 months after random assignment. This variable incorporates information from the BIF 
regarding the level of schooling at baseline. It was coded to ""missing"' if the respondent 
was unable to provide information on attainment of these two credentials at baseline (on 
the BIF) or at follow-up (on the First Follow-up Survey)* It was coded to 0 if the sample 
member (1) already had one or both of these credentials at baseline; (2) did not report 
attending any school or training program of at least a week's duration during the follow- 
up period; and/or (3) reported attending a school or training program of at least a week*s 
duration but did not have either credential at follow-up* Hie variable was coded to 1 for 
those who (1) lacked both credentials at baseline; (2) reported school or training program 
participation during the follow-up period; and (3) had one of the credentials at follow-up. 



Earnings Data from State Unemployment Insurance Agencies 

State agencies responsible for administering the unemployment insurance (UI) program 
collect quarterly data on wages and salaries for most workers* These "wage r^rts" are 
submitted by employers for individual workers, identified by their Social Security 
numbers (SSN). Data obtained from these systems in 14 of the 16 study sites are used 
to test for— and for the adult female target group, compensate for— nonresponse bias in 
the survey data on employment and earnings, as explained in Appendix D* Appendix £ 
contains a comparison of the estimates obtained from these two data sources.^^ 



Content and Collection Method 

Not all employers file wage reports with the UI system. Notable excq)tions include 
federal government and railroad employers, agricultural employers, and the self- 



is. 'Workers* were defined on a month-by-month basis ts those members of the IS-month study sample 
for whom the survey indicates a positive earnings amount. 

16. Usable UI earnings data have not yet been obtained from the states of Ohio or New Jersey. Data 
from a third site are accurate but incomplete, covering only a portion of experimental sample members in 
the site. DaU from this site are therefore omitted from our teau for survey nonresponse bias but included 
in our adjustments to remove bias for the adult women target group and in Exhibit 
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employed. Despite these omissions, an in-depth study of UI earnings data concluded 
recently that "the vast majority of employers are covered in all states/" In most cases 
total earnings are reported for each covered job» including wages» salaries, tips, and 
bonuses. 

Abt Associates obtained UI earnings data through agreements with sute UI 
administrators in IS sites.^* Data requests were submitted to each state at regular 
intervals, usually every six months. Those requests conUined the SSNs of all 
experimental sample members as rq>ortcd on the BIF. The sutes then used SSNs to 
extraa earnings data from the wage records for the sample and return them to Abt. Most 
state response files contain five calendar quarters of data, with a response lag of one or 
two quarters. 

Each response record was matched to the Abt Associates daubase by SSN and, 
where possible, by name and/or date of birth. Incomplete or flawed response files were 
re-requested and any problems discussed with sute staff members. For each SSN states 
would supply zero, one, or many records for each of the quarters covered by the file, 
depending on tlie number of covered jobs held by the individual during the period in 
question. 



Completeness 

As noted earlier this report uses UI earnings dau from 14 of the 16 study sites (all except 
those in New Jersey and Ohio), which together comprise 86 percent of the experimental 
sample. The data span a wide range of calendar quarters and, for most sample members, 
cover several quarters both before -id after random assignment. The data are not 
complete for all sample members in all quarters, however. Because individuals were 
randomly assigned— and their SSNs reported to Abt Associates— over a 23-month period 
(November 1987 to September 1989), not all SSNs were included in the earliest data 
requests. Nor were all calendar quarters fully covered in any particular response file. 
Finally, in rare instances entire response files were unusable, so that calendar quarters 
covered only by those files are completely unavailable. 

Aligned by quarter after random assignment, the resulting dau coverage rates for 
the 14 sites are shown in Exhibit C.2. Only dau for quarters -05 to +06 are used in the 
analysis, however: quarters -05 to -01 as baseline variables and quarters +01 to +04 
as outcome measures in testing for survey nonresponse bias. 



17. Sec Btj, Trott. and Stevens (1991). 

18. An agreement with the sixteenth site is stili under negotiation. 
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Construction of Analysis Variables 

UI earnings data for individual sample members were coll^sed into a set of quarterly 
earnings variables through a series of four steps. First, for each response file, earnings 
across jobs within each quarter were sununed. Second, total earnings for each quarter 
were extracted from the most recent response file that provided data for that quarter. 
Third, zero earnings were imputed for those quarters in which the state provided 
complete data on cases with earnings but no record for the individual in question. And 
finally, the calendar quarter was converted into the quarter relative to random assignment 
(-05 to +06). 
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Exhibit C 1 Imputarion of Missing Earnings and Employment Variables 
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Exhibit C2 Coverage Rcaesfor Earnings Data from State Ul Agencies, 
by Quarter, relative to Random Assignment 
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Appendix D 



Sample Definition and Impact Estimation Methods 



THIS appendix specifies the methods used to analyze outcomes and impacts in 
Chapters 4 through 7, the Executive Summary, and Appendix H. Sections 1-4 
provide information, definitions, and model specifications basic to the analyses of all 
target groups; Sections 5-8 specify the methods used to estimate impacts on adult men 
and youths; Section 9 presents a test for survey nonresponse bias; and Section 10 
describes how we modified the methods of Sections 5-8 to estimate impacts on adult 
women. Section 11 specifies the tests reported in Exhibit H.22 that consider whether 
impacts on the earnings of youths deviate significantly from a linear impact-versus-age 
trend established by estimated impacts on the earnings of adults in their twenties. 



1. The 18-Month Study Sample 

The outcome measures for this repon are taken from the First Follow-up Survey. 
Because the survey was conducted over a 12-month period, while random assignment 
occurred over a 23-month period, the scheduled length of follow-up after random 
assignment varied from 13 to 23 months; the actual length of follow-up varied somewhat 
more because of time lags in locating and interviewing some sample members. In order 
to maintain a constant sample over the period analyzed in this repon, we defined the 
18-month study sample to include only those members of the experimental sample who 
were scheduled to be interviewed at least 18 months after random assignment. (We used 
the scheduled, not the actual, interview date because treatment or control status is 
independent of the scheduled date.) This restriction excluded 3,266 persons, or 15.9 
percent of the ftill experimental sample of 20,601. 
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We also excluded certain experimental sample members who were randomly 
assigned at a different treatment-to-control ratio than the majority of the sample. The 
standard ratio was 2/1. During the course of random assignment, however, five SDAs 
which had difficulty recruiting JTPA applicants were allowed to increase the treatmem-to- 
control ratio temporarily to 3/1 or 6/1 for specific groups. To preserve the balance 
between the treatment and control groups, we randomly selected and excluded from Lhe 
analysis one-third of those treatment group members assigned using a 3/1 ratio and two- 
thirds of those treatment group members assigned using a 6/1 ratio. This procedure 
excluded 473 persons, or 2.3 percent of the fiill experimental sample. 

Finally, we excluded five experimental sample members in Oakland who, according 
to our records, were under age 22 at random assignment. As the Oakland SDA excluded 
youths from the study, these persons either were not intended to be included or were 
older than our records indicate. 

The resulting 18-month study sample includes 17,026 persons, or 82.6 percent of 
the full experimental sample. 



2. Summary of Data Completeness 

A detailed description of the data sources used in this report is given in Appendix C. As 
noted above, the outcome measures are taken from the First Follow-up Survey. The 
other principal data sources for this report are the Background Information Form (BIF) 
and earnings data from state Unemployment Insurance (UI) agencies. Appendix E 
provides a comparison of the UI and survey data on earnings. 

Tne overall completion rate for the BIF was 99.5 percent; the follow-up sui-vey 
response rates by target group and treatment or control status weie: 



In addition to the unit nonresponse of those sample members for whom there was 
no completed BIF or no follow-up interview, there was some item nonresponse on the 
completed forms and interviews. Item nonresponse on the BIF was generally less than 
5 percent. 



Treatment 



Control 



Adult Women 
Adult Men 
Female Youths 
Male Youths 



88.3% 
80.8% 
88.3% 
84.3% 



87.8% 
79.2% 
86.5% 
79.5% 
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Nine-ten^s of one percent of the respondents to the follow-up survey Drovided 
insufficem .nformation to determine employment status in all month's ofX^rov^ul 
period. Because of the low frequency of the problem and the complexity of attemmino 
to salvage mformation on the months for which employment status Luld L deterS*- 
we did not use the survey data on the employment and earnings of these persons.' ' 

When employment status in each month was known but hours worked and/or 
earnmgs m some or all months could not be determined, the missing d ta w re imput^ 
mon4 '"'^^ ^''^ « P- '^-^ of all pers^- 

3. Framework for Statistical Inference 

Our significance tests are based on the assumption that the 18-month study sample is a 
imp e random sample from a much larger population of interest.^ We repon test of 
null hypotheses about the impact (defined below) of assignment to the treafmen group 
on post-assignment outcomes in this population. ^ ^ 

one iIje^S"ed' f™ T "'"^ '° """f'" '^P^'^'^''^^' °^^hich at most 

one IS rwlized for any given person. Suppose we are interested in earnings during some 
period after random assignment. Let denote the amount that person fwt^feaTnTf 

ZTonXl~' ''''''' ''''' ~ '^^^ 

-nie treatment and control means ofy (or treatment and control levels of mean v) 
the population are the population means of and respectively. The Cc" o 
assignment to the treatment group on mean earnings in the population (or iZct per 
assignee) is the difference between the treatment and control means. ' 

number of .niu r.^^oTJ^T '^^^^^^^^^ ""^""^ 'he 

(SUTVA). ^ sublc-unil-ircalmcnl-valuc assumption- 
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The definitions above are appropriate for both continuous and binary outcomes. If 
}• denotes employment status instead of earnings, with values of 1 for employed and 0 for 
not employed, then the treatment and control levels of percentage employed (mean 
employment status) are the population means of yf and yf. The impact of assignment 
to the treatment group on percentage employed is the difference between the treatment 
and control levels. 

Sections 4 and 5 explain how we estimate treatment and control means and impacts 
per assignee and test the null hypothesis of zero impact. 

4. Regression Models: Functional Forms and Regressors 

Unbiased and consistent estimates of treatment and control means and impacts could be 
obtained by simply computing sample mean outcome levels in the treatment and control 
groups and taking their difference. We instead use regression-adjusted means in most 
of our analyses (as indicated in exhibit footnotes) in order to increase the statistical 
precision of our impact estimates and the power of our significance tests. This section 
specifies the models and regressors used; Section 5 gives the details of the basic 
estimation methods. 

We use linear models to estimate treatment and control levels of and impacts on 
mean earnings, hours worked, and weeks worked; we use logistic models to estimate 
treatment and control levels of and impacts on percentage employed. In the notation of 
Section 3, the observed outcome, y,-, is equal to yj if i is a treatment group membe; and 
yl^ if / is a control group member. The linear models assume that the expectation of y,-, 
conditional on the regressors, x,-, is a linear function of x,. The logistic models assume 
that log [p, / (1 - a)] is a linear function of x,, where a is the probability that y^ = 1 (/ 
is employed), conditional on x^. 

The regressors used with each target group consist of a constant, a dummy variable 
for assignment to the treatment group, and a set of baseline covariates shown in Exhibit 
D.l. Where the value of a baseline covariate is missing, we insen the target group 
mean. 



5. Basic Impact Estimation Methods 

This section specifies the methods used to estimate impacts per assignee on most 
outcomes for adult men, female youths, and male youths and for the service strategy 
subgroups and two-year age groups within those target groups. The extensions used to 
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estimate impacts on the components of earnings and on the mean earnings of other 
subgroups are described in Sections 7 and 8, respectively; the derivation of estimated 
impacts per enrollee is explained in Section 6. Section 10 describes how we modified 
these meihods to compensate for the apparent survey nonresponse bias in the data on 
adult women. 



A, Mean Earnings, Weeks Worked, and Hours Worked 

In the analyses of mean earnings, hours worked, and weeks worked, we use ordinary 
least squares to estimate the parameters of the linear model specified in Section 4. The 
regression sample consists of all survey respondents in the target group who provided 
sufficient information to determine employment status in all months of the follow-up 
period. The estimated coefficient on the treatment group dummy variable is our estimate 
of impact per assignee; the two-tailed / test for tiiat coefficient is our test of the null 
hypothesis of zero impact. We estimate the control mean by substituting the target group 
mean covariate values into the estimated model and setting treatment to zero. (In 
calculating the mean covariate values, we include both survey respondents and 
nonrespondents.) The estimated treatment mean is the sum of the estimated control mean 
and impact. 

We estimate separate regressions for each month, for each quarter, and for the 
18-month period after random assignment. 

We estimate treatment and control means and impacts for the two-year age groups 
(Exhibit 6.3) and the service strategy subgroups by estimating a separate regression for 
each of these subgroups, omitting some of the ree-essors in Exhibit D.l when necessary 
to avoid multicollinearity. We estimate the control mean by substituting the mean 
covariate values for the subgroup into the estimated model. 



B. Percentage Employed 

In the analyses of percentage employed, we use maximum likelihood to estimate the 
parameters of the logistic model specified in Section 4. The regression sample is the 
same as described in part A of this section. Again, the two-tailed t test for the coefficient 
on treatment is our test of the null hypothesis of zero impact. The estimation of 
treatment and control levels and impact is more complicated, however. Because the logit 
model is nonlinear, the probability evaluated at the mean values of the covariates does 
not equal the mean of the individual probabilities. If we substituted the target group 
mean covariate values into the estimated logit model and set treatment to zero, we would 
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not get a prediction of what the control group employment rate would be if the treatment 
and control groups had identical baseline characteristics. We would get a prediction of 
what the control group employment rate would be if every control group member had 
covariate values equal to the mean. In a linear model, these two predictions are the 
same, but in a nonlinear model, they generally differ. 

Because the transformation / (I + which converts log odds ratios to 
probabilities, is concave for;c > 0 and convex for;c < 0, substituting the mean covariate 
values into the estimated logit model would tend to give upward biased estimates of rates 
above 50 percent (positive log odds ratio) and downward biased estimates of rates below 
50 percent (negative log odds ratio). 

We therefore adopt the following procedure (Lane and Nelder 1982) to estimate 
treatment and control levels of and impacts on percentage employed. For each person 
/ in the target group, we use the estimated logit model and Vs covariate values to 
calculate two predicted probabilities: the probability that / would be employed if 
assigned to the treatment group; and /5,*^, the probability that / would be employed if 
assigned to the control group. Our estimates of the treatment and control levels of 
percentage employed are the means of /)/ and A^, respectively, over the target 
group.^ The difference between the estimated treatment and control levels is our 
estimate of the impact of assignment to the treatment group on percentage employed. 



C. Distribution of IS-Month Earnings 

The analyses of impacts on the distribution of 18-month earnings for adult men (Exhibit 
5.4) and out-of-school youths (Exhibits H.2, H.8, and H.I5) rely on unadjusted 
frequencies. The nonzero earnings categories are approximate quartiles of the earnings 
distribution of those control group members who had positive earnings. For each of the 
five earnings categories, we report a two-tailed t test of the null hypothesis of zero impact 
on the proportion in that category. We also report a chi-square test of the null hypothesis 
of no impact on the overall distribution (e.g., Snedecor and Cochran 1989, pp. 202-03). 



4. From ihc firsl-ordcr conditions for maximizing ihc likelihood function of ihc logil model, the mean 
of plioT pi^) over all treatment (or control) group members in the regression sample is the unadjusted 
treatment (or control) group employment rate. By uking means over the full Urgcl group, we adjust for 
chance treatment-control differences in baseline characteristics. 



3^7 



JTPA U-MONTH MPaCTS / SAMPLE AND METHODS • 325 



D. Anainment of a Training Related High School Diploma or GED Certificate 

The estimated rates of training-related high school diploma or GED receipt (Exhibits S,5, 
S.ll, 4.9, 5.8, 6.5, 6.10, 7.5, and H.18) are unadjusted percentages. Our two-tailed t 
test of the null hypothesis of zero impact is derived from the unadjusted frequencies 
within the high school dropout subgroup (those who had neither credential at the time of 
application to JTPA); if the null hypothesis of zero impact on this subgroup is rejected, 
we infer a rejection of the null hypothesis for the full sample. Thus, our significance 
levels for the ftill sample are always the same as for the dropout subgroup. 

£, Month of First Job (Out-of School Youths) 

The "month of first job" for out-of-school youths (Exhibit H.23) is defined as follows. 
If the person was employed at any time during the period between the random assignment 
date and the end of the month of random assignment, we let the month of first 
employment equal zero. If the first job after random assignment began during the xth 
month after random assignment, with 1 i^ x <i 18, we let the month of first employment 
equal x. If the person was never employed during the follow-up period, we let the month 
of first employment equal 18. Exhibit H.23 reports unadjusted mean values of this 
variable. We report the standard two-tailed t test for the comparison of the means o( wo 
independent samples. 



6. Impacts per Enrollee: Adjustments for Treatment Group 
nonenrollees and control group crossovers 

For purposes of exposition, we present the adjustment for nonenroUed treatment group 
members (nonenroliT/ first. The estimates of impact per enrollee in this report are, 
however, simultaneou:. , adjusted for noneiirollees and crossovers, as explained in part 
B of this section. The estimates of impact per assignee are not adjusted. 

A. Adjustment for Nonenrollees 

Estimates of impact per assignee do not measure the effect of JTPA on those who 
actually enrolled, because 33.2 to 39.2 percent of treatment group members in each urget 
group did not enroll in JTPA during the first 18 months after random assignment. To 
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esiimate the effect of the program on those who did enroll, we use an adjustment 
proposed by Bloom (1984a)/ 

The impact per assignee is a weighted average of the impact on those who would 
enroll if assigned to the treatment group and the impact on those who would not enroll, 
with weights r and 1 - r, where r is the proportion who would enroll. If we assume thai 
the impact on those who would not enroll is zero, then the impact per assignee is r times 
the impact on those who would enroll. Therefore, to obuin a consistent estimate of the 
impact on persons who would enroll if given the opportunity, we can simply divide the 
estimated impact per assignee by the treatment group enrollment rate. 

Note that the only assumption required fo. this adjustment is that assignment to the 
treatment group has zero average impact on persons who would not enroll. It is not 
necessary to make any assumption about whether enroUees are typical of assignees. 

Unfortunately, the assumption of zero impact on nonenroUees is not innocuous. 
Appendix F describes typical SDA practices regarding formal enrollment in JTPA and 
reports the results of a survey of 307 nonenrolled treatment group members. Roughly 
half of this sample received some JTPA services, although these services were typically 
much more limited than those received by enrollees. 

Under certain conditions, the estimates of impact per enrollee and per assignee will 
estimate upper and lower bounds on the true impact on enrollees. If the true impact on 
nonenrollees is of the sair.a sign as the true impact on enrollees, then the magnitude of 
the estimated impact per enrollee will be an estimate of an upper bound on the magnitude 
of the true impact on enrollees. If the true impact on nonenrollees is smaller in 
magnitude than the true impact on enrollees, then the magnitude of the estimated impact 
per assignee will be an estimate of a loY^er bound on the magnitude of the true impact 
on enrollees. These results follow directly from the expression of the impact per assignee 
as a weighted average of the impact on enrollees and the impact on nonenrollees. 

B. Simultaneous Adjustment for Nonenrollees and Crossovers 

Between 2.0 and 3.8 percent of control group members in each target group enrolled in 
JTPA during the first 18 months after random assignment. Because of these "cross- 



5. The same adjuslment was proposed by Tarwoljo el al. (1987) and Sommer and Zcger (1991) lo 
cslimale the impact of a iherapculic agent on those who comply with their regimen in a randomized clinical 
trial. Sommer and Zcgvir's restriction of their proposal to binary outcomes is unnecessary. 
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overs," average outcome levels in the control group are not unbiased estimates of what 
outcome levels would have been for treatment group members if they had beeu denied 
JTPA services. 

To adjust for the presence of control group crossovers as well as treatment group 
nonenrollees, we used an extension of the adjustment for nonenrollees (Bloom 1985). 
This extension is based on two additional assumptions. First, we assume thai all 
"crossover-type" persons (those who would enroll if assigned to the control group) would 
also enroll if assigned to the treatment group. Second, we assume that, in the notation 
of Section 3, >■,'" and y,^ are equal for crossover-type persons. The impact of assignment 
to the treatment group on crossover-type persons is then zero. 

The impact per assignee is a weighted average of the impacts on three groups: (I) 
those who would enroll if assigned to the treatment group but would not if assigned to 
the control group; (2) those who would never enroll; and (3) crossover-type persons. 
The weights on the three groups are r - c, 1 - r, and c, respectively, where r is the 
proponion who would enroll if assigned to the treatment group and c is the proportion 
of crossover-type persons. Under the assumptions that the impacts on groups (2) and (3) 
are zero, the impact per assignee is r - c times the impact on group (1). Therefore, to 
obtain a consistent estimate of the impact on group (1), we divide the estimated impact 
per assignee by the difference between the treatment and control group enrollment rates 
within the target group or subgroup under study. These enrollment rates are the 
percentages enrolled in JTPA anytime during the first 18 months after random 
assignment. 

This adjustment does not require any assumption that the crossovers are typical of 
the control group. It should be noted, however, that the adjusted impact estimate does 
not apply to enrollees as a whole, but to the slightly narrower population of non- 
crossover-type enrollees. 

Because the crossover rate is low, alternative methods of addressing the crossover 
problem would not change the estimates substantially. 

In Exhibit S.l, estimated impacts on earnings per enrollee are expressed in both 
dollar and percentage terms. The denominator for the percentage calculation is the 
difference between the unadjusted mean earnings of treatment group enrollees in the 
target group and the estimated impact per enrollee. This denominator is an estimate of 
what enrollees would have earned in the absence of the program. 
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7. Percentage Impacts on the Components of Earnings: Estimation Methods 

If an individual is employed at all during a given period, her earnings during that period 
can be expressed as the product of three components: weeks worked, hours worked per 
week worked, and earnings per hour worked. Letting denote earnings, hi hours 
worked, and w,- weeks worked, this statement is simply the equation 

>'/ (hM) (y/^i) • 

Analogously, a group's mean earnings can be expressed as the product of four 
components. Let 7, H, and W denote mean earnings, mean hours worked, and mean 
weeks worked, respectively, with zeros included. Let M denote the proportion of the 
group employed at all during the period. We can express y as the product of four 
components: 

y = M (W/Af) {HIW) {YIH) . 

When percentage impacts on these four components are small, tlie percentage impact on 
mean earnings is approximately equal to the sum of the percentage impacts on the 
components. The first component is simply the percentage employed. The second 
component is mean weeks worked for persons who worked. The third component, mean 
hours worked divided by mean weeks worked, is not necessarily equal to the mean of 
hours per week (A/vv,) for persons who worked. Rather, it is a weighted mean of hours 
per week for persons who worked, with weights proportional to weeks worked. 
Similarly, the fourth component, mean earnings divided by mean hours worked, is not 
necessarily equal to the mean of hourly earnings (yA) for persons who worked; it is a 
weighted mean with weights proportional to hours worked. 

In Exhibits 4.7, 4.14, 5.6, 5.13, 6.8, 6.13, H.4, H.IO, and H.17 we refer to Y 
(mean earnings during the 18-month period) as earnings per assignee and to Af, W/M, 
H/W, and YIH as workers per assignee, weeks worked per worker, hours worked per 
week worked, and earnings per hour worked, respectively. 

We derive estimates of percentage impacts on the four components from the 
estimated treatment and control means of employment status, weeks worked, hours 
worked, and earnings (denoted hereby and , and , fi^ and fl^ , 
and and £^ ). 

The estimated percentage impact on workers per assignee is — - 1 . 
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The estimated percentage impact on weeks worked per worker is 



1^-^ / 



- 1. 



The estimated percentage impact on hours worked per week worked is 



- I. 



The estimated percentage impaa on earnings per hour worked is 



- 1. 



Exhibits S.2 and 7.7 present a decomposition of estimated percentage impacts on 
earnings per assignee into estimated percentage impacts on two components: hours 
worked per assignee and earnings per hour worked. This two-component decomposition 
is completely analogous to the four-component decomposition described above. The 
estimated percentage impact on earnings per hour worked is the same as in the four- 
component decomposition. The estimated percentage impact on hours worked is simply 
the ratio of the estimated impact on mean hours worked to the estimated control mean. 

8. Impacts on Earnings, by Key Subgroup and by Study Snrn 

This section explains how subgroups within each target group were formed and then 
describes the derivation of the subgroup impact estimates and r and F tests reponed in 
Exhibits S.7, S.8, S.13, S.14, S.15, 4.15, 5.14, 6.14, 6.15, 7.8, and H.21. Finally, the 
analysis of impacts by study site in Chapter 7 is described. 

A, Impacts on Key Subgroups Within Each Target Group 

With the exception of the arrest history subgroups of the out-of-school youth target 
groups, all subgroups were defined using information from the random assignment 
telephone file or the Background Information Form. The arrest history subgroups were 
defined using responses to a First Follow-up Survey question about arrests before random 
assignment. When information on a relevant variable was unavailable for cenain 
persons, those persons were not included in any subgroups for which the definitions 
relied on that variable. For example, persons who did not repon marital status were not 
included in any of the household composition subgroups. 

In Exhibits 4.15, 5.14, 6.14, 6.15, and H.21, we present one set of estimated 
control means and tliree sets of subgroup impact estimates. Column (2) gives the 
unadjusted mean earnings of control group members within each subgroup. Column (3) 
gives estimates of the impact per assignee on earnings of each subgroup; these estimates 
are also reponed for selected subgroups in Exhibits S.7, S.8, S.13, S.14, S.15, and 7.8. 
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Column (4) gives model-based extrapolations that estimate what the impact on each 
subgroup would be if the subgroup had the same distribution across sites as the full target 
group. Analogously, column (5) estimates what the impact on each subgroup would be 
if the subgroup had the same distribution across sites and service strategies as the full 
target group. Columns (4) and (5) are shown to examine the question of whether 
differences between subgroup impact estimates in column (3) are due to difTerences in 
distribution across sites and/or service strategies. 

To compute the impact estimates shown in column (3), we estimate one regression 
for each set of complementary subgroups (e.g., the three ethnicity subgroups). Defining 
a dummy variable for membership in each subgroup, we regress earnings in the first 18 
months after random assignment on the subgroup dunmiy variables, the interactions of 
treatment with the subgroup dummy variables, and the baseline covariates in Exhibit D. 1 . 
(The uninteracted treatment group dummy variable is omitted to avoid multicollinearity. 
Certain baseline covariates are also omitted when necessary.) The impact estimates 
shown in colunm (3) are the estimated coefficients on the treatment x subgroup 
interactions. The two-tailed t test on each of these coefficients is our test of the null 
hypothesis of zero impact on mean earnings of the appropriate subgroup. Each subgroup 
impact estimate shown in column (3) converges asymptotically to the same limit as the 
difference in mean earnings between treatment and control group members within the 
subgroup. 

To test the null hypothesis that impacts on complementary subgr-^ups do not differ, 
we perform an F test by estimating a supplementary regression in which the treatment 
X subgroup interactions are replaced by the uninteracted treatment group dummy 
variable. This regression restricts the impacts on the subgroups to be equal. As usual, 
the F test compares the sums of squared residuals from the restricted and unrestricted 
regressions. 

The procedure used to produce the impact estimates shown in column (4) is 
equivalent to a regression of earnings on the regressors use^ for column (3) and 
interactions of treatment with a full set of site dummy variables (omitting one treatment 
X site interaction to avoid multicollinearity). For each subgroup, the impact estimate 
shown in column (4) can be computed by cross-multiplying the estimated coefficients on 
the treatment x site interactions with the target group means of the site dummy variables 
and adding the estimated coefficient on the treatment x subgroup interaction. To 
facilitate the derivation of a t statistic for this estimate, we use an equivalent procedure 
in which treatment is interacted with the deviations of the site dummy variables from 
their target group means. The impact estimate described above is then simply the 
coefficient on the treatment x subgroup interaction, and its t statistic is computed 
automatically by the regression software. We again perform an F test for each set of 
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complementary subgroups by estimating a supplementary regression in which the 
treatment x subgroup interactions are replaced by the uninteracted treatment group 
dunimy variable. 

To compute the impact estimates shown in column (5) and the corresponding t and 
F statistics, we use an analogous procedure, replacing the treatment x site interactions 
with treatment x site x service strategy interactions. 

J5. Analysis of Impacts by Study Site 

The impact estimates and tests reponed in Exhibits 7.9 and 7. 10 are derived by the same 
procedure as the estimates and tests reponed in column (3) of Jie exhibits on key 
subgroups within each target group. Here, the sites are treated as subgroups and site- 
specific impacts are estimated using the treatment x site interactions. 

Additional analyses of impacts by site involved the calculation of correlation 
coefficients across target groups (for Exhibit 7.1 1) and ordinary leasi squares regressions 
of total earnings during the I8-monlh period on a set of covariates and treatment x 
covariate interactions described in Chapter 7 (see discussion of Exhibit 7. 12). The usual 
formulas were used to calculate the correlation coefficients, treating each site as an 
observation. 



9. Test for Survey Nonresponse Bias 

The overall response rate for the First Follow-up Survey was 84.8 percent, with target 
group response rates ranging from 80.3 to 88.2 percent. We tested for survey 
nonresponse bias in the impact estimates, using Unemployment Insurance (UI) earnings 
data for both survey respondents and nonrespondents in 13 of the 16 study sites 
(excluding Bune, Montana; Jersey City, New Jersey; and Marion, Ohio). To construct 
a test, we used two subsets of the 18-month study sample to estimate impacts on the sum 
of Ul-reported earnings over the first four calendar quarters after random assignment: 
(1) all members with complete UI earnings data for calendar quarters M; and (2) all 
members in the first group who also had complete survey earnings data for months 1-18. 
These rmpact estimates were produced by the estimation method described in Section 5, 
a linear regression of total Ul-reported earnings over calendar quarters 1-4 on a treatment 
group dummy variable and the regressors in Exhibit D.l. The difference between the 
two impact estimates is an estimate of the bias introduced by restricting the analysis to 
survey respondents. 
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The estimated bias, by target group, is given below. 

Estimated Bias per Year 

Adult Women $86 

Aviult Men -$26 

Youths -$38 

The estimated biases for adult men and youths were judged acceptable, while the bias for 
adult women was deemed unacceptable. Therefore, in the analysis of impacts on the 
earnings and employment of adult women, we applied- imputation procedures described 
in the next section. 

The nonresponse bias fur adult women appears to be concentrated in the treatment 
group. The unadjusted means of total Ul-reported earnings over calendar quarters M, 
by treatment or control status and response or nonresponse, were: 

Treatment Control 
Respondents $4,154 $3,695 

Nonrespondents $2,950 $3,481 

10. Adjustments for Survey Nonresponse Bias in the Data on Adult Women 

This section specifies the imputation procedures used for the adult female target group 
in estimating treatment and control levels and impacts on mean earnings, the distribution 
of earnings, percentage employed, mean weeks and hours worked, and the components 
of earnings. The general approach was to use Unemployment Insurance QJl) earnings 
data to impute individual or mean outcome values for survey nonrespondents (and for 
respondents who provided insufficient information to determine employment status in all 
months) in 14 of the 16 study sites (excluding Jersey City, New Jersey, and Marion, 
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Ohio).^ At the end of this section, estimates based on the imputations are compared with 
estimates that use data on respondents only. 



A. Mean Earnings 

To impute earnings for nonrespondents, we first estimated 18 linear regressions (one for 
each month) with the subset of adult female respondents that had UI earnings data 
covering any of the first six calendar quarters after random assignment. In this 
subsample, we regressed survey-reported earnings on a constant and 18 additional 
regressors, three for each of calendar quarters 1-6: a dummy variable indicating whether 
Ul-reported earnings were missing; a dummy variable indicating whether Ul-reported 
earnings were greater :han $5,000; and a continuous variable set equal to zero if Ui- 
reported earnings were missing or greater than $5,000 and set equal to Ul-reported 
earnings otherwise. We used the estimated coefficients from these regressions to predict 
the missing values of survey-reponed earnings in each month for survey nonrespondents 
who had UI earnings data covering any of calendar quarters 1-6, The nonrespondents 
were then included, with these predicted values, together with the respondents in linear 
regressions of the form specified in Sections 4 and 5, To protect our significance tests 
for impacts on mean earnings against any conditional heteroskedasticity introduced by the 
imputations, we used the White (1980) standard error estimator. 



B. Distribution of J 8-Month Earnings 

In Exhibit 4.5, the nonzero earnings categories are approximate quartiles of the earnings 
distribution of those respondents in the control group who had positive earnings. We 
added imputed frequencies of nonrespondents to unadjusted frequencies of respondents. 
The imputed frequencies of nonrespondents were derived by the following method: 

We divided the range of total Ul-reported earnings over calendar quaners 1-5 into 
thineen intervals: $0; eight $l,000-width intervals (from $1-1,000 to $7,001-8,000); 
three $2,000-width intervals O^eginning at $8,001; $10,001; and $12,001); and $14,001 
or more. 



6. Because Ihc UI records from Montana appeared accurate but covered only a subset of all sample 
members in the site, we included Montana in these imputations but not in the test for nonresponse bias. 

7. The number of adult women with Ul-reporicd earnings exceeding $5,000 in a quarter ranged from 
23 in the first quarter to 96 in the sixth quarter. The number of observations in each regression was 4,965, 
or about 85% of all adult female respondents. The ranged from .35 to .42, except in the regression for 
the first month, which had an of .26. 
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Examining the set of persons for whom both survey and UI earnings data were 
available, we produced a 13 x 5 contingency table of total Ul-reported earnings over 
calendar quarters 1-5 by total survey-reported earnings over quaners 1-6, using the 13 
UI earnings categories listed above and the 5 survey earnings categories shown in Exhibit 
4.5. We used this contingency table and the distributions of Ul-reported earnings for 
nonrespondenis in the treatment and control groups to predict how many nonrespondenis 
in each of the two groups would fall into each of the five survey earnings categories. 

The r and chi-square tests reported in Exhibit 4.5 treat the adjusted frequencies as 
if they were unadjusted. 



C. Percentage Employed, Mean Weeks and Hours Worked, and the 
Components of Earnings 

To impute employment status for nonrespondents, we first estimated seven logistic 
regressions (one for each quaner and one for the 18-month period) with the subset of 
adult female respondents that had UI earnings data covering any of the first six calendar 
quarters after random assignment. In this subsample, we regressed survey-reported 
employment status on a constant and 12 additional regressors, two for each of calendar 
quarters 1-6: a dummy variable indicating whether Ul-reported earnings were missing 
and a dummy variable indicating nonzero Ul-reported earnings. We used the estimated 
coefficients from these regressions to calculate predicted probabilities of employment in 
each period for survey nonrespondenis who had UI earnings data covering any of 
calendar quarters 1-6. A random number generator was used to impute employment 
status in each period for these nonrespondents. The nonrespondenis were then included, 
with these imputed values, together with the respondents in logistic regressions of the 
form specified in Sections 4 and 5. 

Adjustments for nonresponse bias were then extended to estimates of impact on 
weeks and hours worked, which could not be observed in the UI data. Here, we applied 
adjustments that make estimated percentage impacts on mean weeks and hours worked 
compatible with the estimated percentage impacts on employment status and earnings.* 
We began by using the formulas shown in Exhibit D.2. 



8. V/c refer here lo pcrcenUgc impacts on weeks tnd hours worked averaged across all sample 
members, including those persons who did not work during the follow-up period. As described below, later 
steps in the procedure convert these percentages into natural units (weeks and hours) and use them to 
calculate percentage impacts on weeks worked per worker and hours worked per week. 
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The formulas in Exhibit D.2 were designed for the cases in which b and c lie 
between a and d: that is, in the respondent-based set of estimates, the percentage impacts 
on weeks and hours worked lie between the percentage impact on percentage employed 
(the impact on percentage employed expressed as a percentage of the control level) and 
the percentage impact on earnings. The formulas then use the relative positions of b and 
c in the interval between a and d to impute the relative positions of the nonresponse- 
adjusted estimates of percentage impact on weeks and hours worked in the interval 
between x and z (the nonresponse-adjusted estimates of percentage impacts on 
employment and earnings). This approach translates the progression of growing (or 
declining) percentage effects on employment, weeks worked, hours worked, and earnings 
for respondents (a, Z?, c, d) into a matching progression between two different 
nonresponse-adjusted endpoints (x and z). 

The formulas in Exhibit D,2 are not appropriate when b or c lies outside the inten/al 
bounded by a and d. In such cases, we allowed the corresponding nonresponse-adjusted 
estimate to lie above or below the interval bounded by jc and z by the same percentage 
point amount. Thus, for example, if c > d > then the nonresponse-adjusted estimate 
of percentage impact on hours worked was set to z + (c - ^. 

A third procedure was used when d — a and z — x took opposite signs. This 
occurred only for the other services strategy subgroup in quarters 1 and 2. In this 
instance, we concluded that the positions of b and c relative to a and d in those quaners 
were of no use in imputing nonresponse-adjusted estimates of impact on weeks and hours 
worked. We imputed estimates of impaa on weeks and hours worked in those quarters 
by subtracting the sum of the nonresponse-adjusted impact estimates for quarters 3-6 
(which were all derived by the first or second procedure) from the nonresponse-adjusted 
estimate of impact for the entire follow-up period (which was derived by the second 
procedure) and dividing the difference evenly between quarters 1 and 2. 

We estimated the control mean of weeks worked by multiplying the respondent- 
based control mean of weeks worked by the ratio of the nonresponse-adjusted control 
level of percentage employed to the respondent-based control level of percentage 
employed. To estimate the control mean of hours worked, we multiplied the respondent- 
base-i control mean of hours worked by the ratio of the nonresponse-adjusted control 
mean of weeks worked to the respondent-based control mean of weeks worked. We then 
derived estimates of treatment means of and impacts on weeks and hours worked from 
the new control means and the nonresponse-adjusted estimates of percentage impact 
described above. Our significance tests for the nonresponse-adjusted estimates of impacts 
on weeks and hours worked use the estimated standard errors for the respondent-based 
estimates. 
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Finally, we derived estimates of impacts on the components of earnings by applying 
the procedures described in Section 7 to the nonresponse-adjusted treatment and. control 
means. 



D, Comparison of Results With and Without Nonrespondent Imputations 

Exhibit D.3 shows two sets of estimated control levels of and impacts on mean earnings 
of adult women. The estimates in the top panel use data on respondents only. The 
estimates in the bottom panel, which are also presented in Exhibits 4.3 and 4.4, are 
derived by the procedure described in part A of this section.* Note that although the two 
estimates of the control mean of total earnings over the follow-up period differ by only 
$19, the respondent-based estimate of impact over the follow-up period is $106 higher 
than the nonresponse-adjusted estimate. The nonresponse bias adjustment lowers the 
treatment mean but hardly changes the control mean because, as reported in Section 9, 
Ul-reported earnings of nonrespondents in the treatment group are substantially lower 
than those of treatment group respondents, but Ul-reported earnings of respondents and 
nonrespondents in the control group are similar. 

Exhibit D,4 shows the respondent-based and nonresponse-adjusted estimates of 
control levels of and impacts on percentage employed, mean weeks worked, and mean 
hours worked. The nonresponse bias adjustments tend to lower the impact estimate, but 
the effect on the control mean varies. 



11. Test for Deviations of Impacts on Youth Cohorts from Trend 
IN Young adult Impacts 

As noted in Section 5, the estimated treatment and control means and impacts for the 
two-year age groups shown in Exhibit 6.3 are derived from split-sample regressions. 
Exhibit H.22 reports the results of tests designed to consider whether impacts on the 
youth age groups deviate significantly from a linear impact-versus-age trend established 
by the adult age groups. We perform these tests separately for females and males. We 
estimate the following equation, pooling all 18-month study sample members of the 
relevant gender who were age 16 to 29 at random assignment: 

y. = a + b'x, + cA, + dKi + ^L, + gM^ + T,(p + qA, + ccK, + )3L, + yM,) + c„ 



9. All significance tests reported in Exhibit D.3 rely on the White (1980) standard error estimator, 
although use of the conventional estimator would not have altered significance levels appreciably. 
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where 

>'f = total earnings during the first 18 months after random assignment, 
X, = a vector of baseline characteristics; 

Ai = an age •'counter" for two-year age groups, beginning at ages 28-29 (/4,= 1) 
and continuing to ages 16-17 (/4,=7); 

Ki = 1 if age 16-17 at random assignment, 0 otherwise; 

L, = 1 if age 18-19 at random assignment, 0 otherwise; 

Mi = 1 if age 20-21 at random assignment, 0 otherwise; 

and 

7] = 1 for treatment group members, 0 for control group members. 

The expression in parentheses models impact as a linear function of the age counter 
and the dummy variables for the youth age groups. If impacts on the earnings of youths 
follow the impact-versus-age trend established by impacts on the earnings of adults, then 
a=)3=7=0. 

The vector used consists of all regressors listed in the youth column of Exhibit 
D-1 except the age dummies. We estimate the model by regressing earnings on a 
constant, x„ K^, L-, M^, T^, and the interactions of 7] with A^, K^, L„ and M,. We then 
perform / tests of the three null hypotheses a=0; i3=0; and 7=0. The estimates of a, 
)3, and 7 are reponed in Exhibit H.22. 



4mo 



338 TTPA 18-MONTH IMPACTS / SAMPLE AND METHODS 



Exhibit D.J Baseline Characteristics Used as Regressors in Adjusting 

for Differences Between the Treatment Group and 
Comrol Group, by Target Group 

Regressor Adult Adult Oui-of'School 

Women Men Youilu ^ 

Ethnidty 

(While, non-Hi sptnic) 

Black, non-Hisptnic 

Hispanic 

Asian, Pacific Islander, American 

Indian, or Alaskan Native 

Education and training histories . 

Adull Basic Education or ESL 

High school diploma 

GED certificate 

Some college 

Occupational training 

Technical certificate 

Job search assistance program 



Work histories 
Ever employed 
Emnloyed upon application 

c 

Total earnings in past 12 months 
Hourly earnings in most recent job 



Weeks worked in past 12 months 
(Zero) 
1-26 weeks 
27-52 weeks 



0-27 wveks 
a8-52 weeks) 



Ul-reported earnings in each of 5 
quarters before random assignment^ ' ^ 

Public assistance histories 

Receiving Food Stamps 

Receiving cash welfare other than AFDC 

Receiving unemployment benefits 

Any unemployment benefits, past 12 months 



(conthvied) 
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ExJubir D. I 



Baseline Charaaerisncs Used as Regressors in Adjushn^ 
for Differences Between the Treatment Croup and 
Control Croup, by Target Group (Continued) 



AFDC histories 

Ever AFDC case head 

Case head anyiiine in past 12 months 

Received AFDC all of pasi 12 months 

Years as AFDC case head: 

(Less than 2 years) 

2-5 years 

More than 5 years 



8 



JTPA required 
Required to tpplv 

Household composition 
Marital sutus: 

(Never married) 

Spouse present 

Widowed, divorced, or separated 

h 

Number of persons in household ^ 
Number of own children present 
Own child under age 6 present 

Fa0uly income in past 12 months 
Uss than S3,(X)0 
$3,O00-S6,00O 
(More than S6,(X)0) 

Living in public housing 
Yes 

Transportation / communication 
Driver's license 
Car available for regular use 
Telephone at home 

Age at random assignment 

(16-19) 
20-21 

22-29 
30-44 
45-54 

(55 or older) 



Adidi 
Women 



AduH 
Men 



icotumucdj 
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Exhibit D. 1 Baseline Characteristics Used as Repressors in Adjusting 
for Differences Between the Treannent Group and 
Conirol Group, by- Target Group (Continued) 



Racomraciided pro(;nun services j 
6 dummy variables, not mutually cxclu>ivc 

Site k 
15 mutually exclusive dumm> variables 
(1 dummy variable omitied) ^ 

14 mutually exclusive dummy variables 
(2 dummy variables omitied) 

Sampling cohort 

ModUis between date site began nu)dom ^ 
assigmneot and date individual assigned: 

(Less than 7 monihs) 

7- 12 monihs 

More than 12 monihs 



Adult 
Women 



Adult 
Men 



Out'Of'Sdtool 
Youtlis 



Key: [ [ Used as a regrcssor 

I I Not 'jscd as a regrcssor 



Notes: Each regrcssor is a dumnr-;. variable, excq>l where noted otherwise. Variable* in parentheses are 
omitied from ihe regressions to avoid exact colhncarily. 

Source: Backgrour^ Infonmalion Fonm responses, except where noted otherwise. 
CL The same set of regrcsson is used for both female and male oul-of- school youlhs. When pooling 
females and males, we add a dummy variable for gender. 

b. English as a Second Language. 

c. A continuous (not a dummy) variable. 

d. Set equal to a constant if nc\'cr employed. (Because a dummy variable for "ever employed" is also included, 
the particular constant chosen is irrclcvanl to the impact estimates.) 

<f. Source: Earnings data from state Unemployment Insurance (Ul) agencies. 
/ General Assistance. Home Relief, or any other welfare thai is not AFDC or Food Stamps. 
^. A? a reiquiremcnt to receive welfare. Food Stamps, or as pan of the federal Work Incentive (WIN) program; 
or by a court order, 

h, A count (not a dummy) variable. 

i. Sources: Background Information Form responses (for date of b' ih) and random assignment telephone file 
(for date of random as(<Knmcnt). 

The six service categories arc: classroom training in occupational skills; on-the-job training; job search assistance; 

basic education; woric experience; and miscellaneous services, 
t Fifteen site dummy variables arc used with the adult urgei groups; 14 are used with the youth target groups, 

which have no l8-monih study sample members m Oakland. 
/. Source: Random usignmcnt telephone file. 
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Exhibit D. 2 Derivation of Estimated Percentage Impacts on the Number of V/eeks 

and Hours Worked, Adj listed for Sur\ r\' Nonresponse Bias 

Impact as a percentage of control ^roup mean 

Outcome measure Estimate before adjustment Estimate adjusted lor 

(by quarter or over (based on respondents) nonresponse bias 
all quarters) 

Percentage employed at a x 

any time during period 

Weeks worked during period b a:+ (b-a) [(z-x )/ (d-ajj 

Hours worked during period c a + (c-a) {(z-x )/ (d-a)J 

Earnings during period d z 



4^4 
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Exhibit D,3 Estimated Impacts on Earnings, with and 
without Nonrespondent Imputations: 
Adult Women 





Control 


impact per 


Period 


mean 


assignee, in S 




(I) 


(2) 




Respondents only 




Quarter 1 


$ 911 


$ 37 




1,139 


79** 


•J 


1,235 


140*** 


4 


1,376 




5 


1,427 




6 


1,425 


160*** 


All quarters 


7,507 


645*** 




With nonrespondent imputations 


Quarter 1 


$ 916 


$ 26 


2 


1,145 


60* 


3 


1,236 




4 


1,363 


78** 


5 


1,413 


116*** 


6 


1,414 




All quarters 


7,488 





Source: Top panel, esumaies based on First Follow-up Surv'e> responses; 
bonom panel, esiimaics based on Firei FoUow-up Survey responses 
and earnings daia from stale UI agencies. 

Notes: Sample size, lop panel, assignees = 3.881; control group = 1.844. 
BoUom panel, assignees = 4,376; control group = 2,098. 
• Suiisiically significant at the .10 level, at the .05 level, 
at the .01 level (iwo-uiled tesi). 
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Exhibit D.4 Esiiniated Impacts on Employment, with and without Nonrespondent 
Imputations: Adult Women 





Percentage et 


rployed 


Weeks worked 


Hours 


worked 




Control 


Impact 


ContrOi 


Impact 


Control 


Impact 


Period 


mean 


in 9c 


tnean 


in weeks 


mean 


in hours 




(1) 


(2) 


(3) 


(4 1 


<5) 


(6. 








Respondents only 






Quarter 1 


47.97c 


0.4 9J 


5.0 


0.0 


181 


0 




53.2 


1.6 


6.0 


0.1 


220 


4 


3 


56.1 


2.4* 


6.3 


0.3* 


233 


14** 


4 


60.1 


1.0 


6.8 


0.1 




9 


5 


59.5 


3.9*** 


7.0 


0.3** 


258 


15** 


6 


61.0 


3.3** 


7.0 


0.5*** 


254 


JO*** 


All quarters 


76.4 


2.6** 


38.1 


1.3* 


1.397 


62* 








With nonrespondeni ifnputations 






Quarter 1 


48.4% 


0.09c 


5.1 


0.0 


183 


-1 




53.4 


1.4 


6.0 


0.1 


220 


3 


3 


55.8 


2.4* 


6.3 


0.3* 




12^^ 


4 


59.7 


0.8 


6.8 


0.1 


250 


8 


5 


59.3 


3.3*** 


7.0 


0.3 


257 


12* 


6 


60.9 




6.9 


0.3** 


254 


17** 


All quarters 


76.8 


2. 1** 


38.3 


1.1 


1,403 


52 


Source Top pane]. e«timaie< based on Pint FoIlou/-up Survey resporues; boOom 


panel, ectimaiet baxed on Firel 




Fir« Follow-up 


Survey re«ponte« and eamingt 


data from ctaie Ui agencies. 








Noiec: Sample size, top panel. a4signe«< > 3. 


}8l; control 


group - l.t44. Botum panel. a*sifne«< - 4.376; control 




group ° 2.098. 


* Suuitically significanl al th< 


.10 level. • 


• ai ihe .05 level. 


althe .01 level (two*uil<d t«iJ. 
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Appendix E 



A Comparison of Earnings, Employment, and 
Impact Estimates Based on Data from the First 
Follow-Up Survey and from State Unemployment 

Insurance Agencies 



THIS appendix compares follow-up earnings and employment estimates, as well as 
impact estimates, from two data sources: First Follow-up Survey resionsrand 

mTin ""r^^^^"^" agenciJ^-' ^ieViT^^^^^^^ 

Survey ,s the mam source of impact data for the present repon, and UI earnings records 

^e^r:inT.::T1 °' '""''V''' °" repon. Hence, the compaSSy o 
the earnings and employment information from these two sources is an issue of central 

Te^sTt ^Jestr^h''"'!,'™ ^^P^"^''' ^^^"^ b' f re g'n 

interest to researchers who must choose between the two types of data sources in future 
evaluations of employment and training programs. 

rh. .I?H this appendix are for a special sample constructed solely to compare 

the wo data sources. Hiis 12-month comparison sample includes all treatment group and 
Zl!Tr:TZ^ 18-month study sample for whom ..2^!^:^. 
available from both data sources for the first four calendar quarters after random 
assignment. Because DI earnings data are not currently available'from New Jersey and 



1. Sec Appendix C for a dcscripiion of these two dau sources. 



345 



346 ♦ JTPA 18-MONTH IMPACTS /SURVEY-UI DATA COMPARISON 



Ohio, sample members from the two study sites in those states— Jersey City and 
Marion— are not included in the analysis here. 

We limited the analysis to the first four calendar quarters after each sample 
member's random assignment, because it was that period for which the most survey and 
UI data were available and because UI records report earnings only by calendar quarter. 
The sample is therefore constant across all four follow-up quarters. Because this 
comparison sample is a subsample of the 18-month study sample, none of the survey- 
based estimates presented below match those elsewhere in the report. 

Our comparison focuses on the only two outcome measures currently available from 
the UI data set: quarterly earnings, and quarterly employment.' The survey-based 
earnings measure is defined as total quarterly earnings from wages, salaries, tips, and 
bonuses.^ The Ul-based earnings measure is defined as all types of earnings from all 
jobs reported to the state's unemployment insurance system. For both data sets employed 
individuals are defined as those with positive earnings in the period. 

Several factors may cause the earnings and employment estimates from the two 
sources to differ. First, although UI earnings records cover over 90 percent of all 
workers, they do not report on the earnings of persons who are self-employed or who are 
railroad, federal, or out-of-state employees. Second, UI records may miss unreported 
earnings from casual work, the underground economy, or tips from reported jobs. 
Third, they may report severance pay as regular earnings. And finally, UI records may 
contain random reporting ersOrs and incorrect social security numbers. 

On the other hand, the survey-based estimates may contain other errors stemming 
from recall problems. Some survey respondents may have forgotten to report a particular 
job or may have inflated their earnings in the interview, although they had no obvious 
incentive to do so. Random reporting or coding errors could also occur in the survey 
data. 

Within any quarter we observe, the earnings estimates from the ivvo data sources 
may also conflict because of timing differences. The survey respondents, for example, 
may have misstated the start date or end date of a job spell whereas employers, 
particularly small businesses, may have reported earnings to UI agencies with a lag, 
making the UI records for a given quarter incomplete. Moreover, UI records report 



2. Ail extension of this analysis in our final report will incorporate a variable indicating the number of 
jobs reported caeh quarter. 

3. The survey-based nncasureof quarterly earnings excludes earnings from so-called odd jobs reported 
in the First Follow-up Survey, Odd-job earnings during the follow-up period were not part of a specific job 
spell. 
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earnings when they are paid, not earned, whereas survey respondents may not have 
followed this convention. 

Thus, the possibilities of differences in reporting errors from the two data sources 
are numerous, leading to a potential for differences in the earnings and employment 
estimates calculated from each source. If the reporting differences are random, they will 
not bias the estimates of program impacts. But if the reponing difference's are 
systematic, and especially if they differ between the treatment group and the control 
group, the two data sources will yield different impact estimates. 

The analysis described below does in fact find consistent differences between the 
earnings and employment estimates based on First Follow-up Survey responses and those 
based on UI earnings data, but it was not possible to identify fully the reasons for those 
differences. An expanded version of this analysis to be presented in our final report will 
explore the reasons in more detail. 

For now, however, our principal findings are twofold: 

• The survey-based estimates of average earnings are consistently and 
substantially higher than the Ul-based estimates. 

• Nevertheless, the estimated JTPA impacts on earnings are similar for the 
two sets of estimates, in the cases of adult women, adult men, female out- 
of-school youths, and most male out-of-school youths. The main exception 
to this rule was the subgroup of male youths who had been arrested between 
their sixteenth birthday and when they applied to JT (25 percent of all 
male youths in the treatment group). For this subgroup the survey data 
indicated a large negative impact, whereas UI data indicated a negligible 
impact. 

The remainder of this appendix details the findings from the analyses. 



Differences in Earnings Estimates 

This section reports the results of four separate analyses of differences in the earnings 
estimates based on the survey and UI data. The first analysis, of differences in the 
earnings distributions estimated from each data source, establishes the pattern of 
relatively higher survey-based than Ul-'uased earnings estimates. The other three analyses 
investigate the possibility of site- or time-specific reasons for this pattern. 
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Distributions of 12-Month Earnings 

A simple way to compare earnings estimates based on the two data sources is to compare 
total reported earnings for all four quarters combined. Our comparison therefore begins 
with frequency distributions of the mean earnings of the 12-month comparison sample 
(including the treatment and control groups) over the first year after random assignment. 
As shown in Exhibit E.l, the UI data reported more sample members earning under 
$10,000 than did the survey. Correspondingly, the survey data show more sample 
members in the higher earnings categories up to $40,000 annually. These findings are 
consistent acros** all four target groups. 

The tendency for the earnings distribution based on the survey data to lie above the 
earnings distribution based on the UI data is offset somewhat by a slightly larger 
proportion of sample members with zero 12-month earnings in the survey data than in 
the UI data. 



12-Month Earnings by SrrE 

In a second analysis we examined whether the finding of relatively higher survey-based 
earnings estimates holds true across the 16 study sites. If the survey-based estimates 
were higher than the Ul-based estimates in only a few of the sites, we would then need 
to explore site-specific reasons for the discrepancies. 

Exhibit E.2 presents a survey-UI comparison of mean annual earnings, by site, for 
each target group. The ratio of the survey-based estimate to the Ul-based estimate for 
each target group, by site, (column 3) serves as a simple summary statistic with which 
to compare discrepancies across the sites for which we have data. The findings are clear. 
The survey-based averages exceed the Ul-based averages by a factor ranging from 35 
percent for adult women overall to 80 percent for male out-of-school youths. This 
pattern is consistent across sites within each target group.^ 

This finding of consistently higher survey-based estimates across the sites rules out 
problems arising from data collection errors in only a few specific sites— or types of jobs 
found predominantly in only a few sites— as the ."ource of the discrepancies. We 
therefore must consider more general explanations, si- :h as erroneous reporting of job 
spells or earnings by the survey respondents or the widespread occurrence of jobs or 
types of earnings that tend to be missed in UI earnings reporting. 



4. In each target group panel in the exhibit, the magnitude of the outlier ratio displayed in the first row 
(for instance, 4.66 in the case of adult women) prompted further exploration, which indicated that the UI 
earnings data for that site were incomplete. Revised data on the site arc not yet available. 
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12-MoNTH Earnings, by Location Near a State Border 

The consistency of the pattern across sites also suggests that out-of-state jobs held by 
sample members are not a major source of the difference between the survey- and UI 
based estimates. If sample members received a substantial amount of their earnings from 
employers outside the state in which they lived when they applied to JTPA (for example, 
because they commuted to out-of-state jobs or subsequently moved to a neighboring 
state), one would expect the survey-based earnings estimates to exceed the Ul-based 
estimates by substantially more in those sites that were near a state border. 

But as shown in Exhibit E.3, the ratio of survey to Ul-based earnings estimates 
dififered little between sites near a state border and sites not near a border. Indeed, this 
ratio was slightly smaller, not larger, in those sites near a state border.^ 



Quarterly Earnings 

The next step in our analysis was to determine whether the finding of relatively higher 
survey-based than Ul-based earnings estimates was constant over all four quarters after 
random assignment. One might expect, for example, that the incidence of survey recall 
errors would decrease in more recent quarters, whereas lagged reporting by employers 
might cause earnings to be under-reported to a greater extent in more recent quarters. 
Moreover, as time went on, sample members may have been more or Jess likely to find 
the types of jobs that would be reported in one but not the other data source, such as 
federal jobs or self-employed work. 

Exhibit E.4 displays estimates of mean earnings in each follow-up quarter, based on 
each of the two data sources. Again, the survey-UI ratio of these means is a convenient 
summary statistic to examine. The clear message of Exhibit E.4 is that the pattern of 
relatively higher survey-based estimates is stable across all of the first four quarters after 
random assignment. 

The exhibit also shows that the discrepancies between the two data sources are 
statistically significant at the .01 level.* We therefore need an explanation of the 
relatively higher survey-based estimates that is relevant for all follow-up quarters. 



5. A site was classified as near a sUlc border if ii included any counties adjacent to another sute. 

6. Although Exhibit E.4 shows the ratio of the two earnings estimates, we tested whether this ratio was 
significantly different from 1 (one) by employing a two-uilcd t-tcst of the hypothesis that the paired 
differences between the survey and Ul-bascd earnings was zero. 
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Differences in Estimated Employment Rates versus Differences in 
Estimated Earnings per Employed Sample Member 

To look beyond simple comparisons of total earnings, we decomposed the survey-UI ratio 
of total earnings into the product of two components: the survey-UI ratio of employment 
rates, and the survey-UI ratio of mean earnings per employed sample member. If most 
of the survey-UI difference in total earnings arose from differences in the quarterly 
employment rates reported, any explanation of the larger survey-based estimates of total 
earnings should stress factors such as jobs that the survey but not the UI records 
reponed, delayed reporting of jobs in the UI data, or exaggerated lengths of job spells 
in the survey data. 

On the other hand, if most of the survey-UI difference in total earnings arose from 
differences in reported earnings per employee, we should suspect such reasons for the 
discrepancy as differences in the types of jobs each data source reponed, tip income or 
second jobs reported only by the survey data, or exaggeration of income or job spell 
lengths in the survey data. 



Employment Rates 

As the first step in this analysis Exhibit E.5 presents quarterly and 12-month employment 
rates obtained from the two data sources, by target group. The percentage employed is 
defined as the percentage of the sample with any recorded earnings during the period in 
question. The exhibit indicates that the employment rates based on the two data sources 
are not nearly as different as the earnings estimates were shown to be in the preceding 
section. For each target group the survey-based employment rates were actually slightly 
lower in the first quaner and for all four quarters overall than the Ul-based employment 
rates. But in each target group the survey-UI ratio increased over the last three quaners. 
In the fourth quarter the survey-based employment rate exceeded the Ul-based rate by 
7 percent to 10 percent for females (column 3) and by 16 percent to 19 percent for males 
in the sample (column 6).'' 

The main lesson of Exhibit E.5 is therefore that the relatively higher survey-based 
than Ul-based estimates of total earnings are not largely attributable to higher survey- 
based employment rates. 



7. The raiio of survey-based lo Ul-bascd employmcni rales (or mean earnings) provides a dircci 
indicaiion of ihe percentage by which ihe survey-based esiimaie exceeds or falls short of ihe Ul-bascd 
cslimaie. For example, if the raiio is 1 .07, the survey-based esiimaie is 7 percent hrger than the Ul-based 
estimate; and if the ratio is 0.93, the survey-based estimate is 7 percent smaller than the LM-based estimate. 
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Nevertheless, we still need an explanation for the fourth quarter findings. One 
possible explanation for the higher survey-based rates in the fourth quarter is employer 
delays in reponing earnings to state UI agencies. If some employers repon the earnings 
of their employees to the state later than others, then at any given time the state UI 
records that are most incomplete are those for the most recent quarter. Thus, the UI 
earnings data we obtained from the states would be less complete for the most recent 
quarters. This would yield a pattern in which the Ul-based estimates of earnings and 
employment understated true earnings and employment to a greater extent in the later 
follow-up quarters. 

One way to explore this possibility is to examine the Ul-based employment rates of 
the subsample of the 12-month comparison sample with six quarters of complete UI 
earnings data. If reporting delays were a problem, one would expect to see higher 
reported employment rates during the first four follow-up quarters for die six-quarter 
subsample than for the 12-month comparison sample. But as shown in Exhibit E.6 there 
is no consistent difference between the Ul-based employment rates for the six-quarter 
subsampie and those for the full comparison sample. Thus, there is no evidence of 
reporting delays in the UI earnings data used in the present analysis. 

Quarterly Earnings per Employed Sample Member 

Since the survey-UI discrepancies in employment rates are fairly small, we must look to 
the second component of total earnings for some insight into the issue. Exhibit E.7 
displays the mean earnings of employed sample members, by target group, during each 
quarter and over all four quarters. The results here confirm that most of the difference 
between the survey and Ul-based estimates of total earnings reflects the fact that the 
survey reported higher earnings per employed sample member. Moreover, the survey-UI 
ratios for earnings per employee over all four quarters are very close to the ratios for 
total 12-month earnings, shown in Exhibit E.4. Nevertheless, the survey-UI ratios of 
earnings per employee in Exhibit E.7 fall steadily from the first to the fourth quarter after 
random assignment, for all four target groups. 

Indeed, when we examine Exhibits E.4 through E.7 together, an interesting time 
pattern emerges. The constancy of survey-UI ratios of total earnings over time is the 
result of two countervailing trends: First, the survey-UI ratios of employment rates rise 
over time, and second, the survey-UI ratios of earnings per employee fall over time. 

We can only speculate about the forces underlying these trends. For example, 
survey-UI ratios of employment rates would rise over time if survey respondents were 
less likely to forget more recent jobs. An explanation of the falling survey-UI ratios of 
earnings per employee is harder to come by, however. Perhaps respondents were more 
likely to exaggerate earnings or job spell lengths when recalling jobs in the more distant 
past. 
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Furtiier Analysis of Differences in Earnings 
per Employed Sample Member 

To learn more about the survey-UI discrepancies in estimated earnings per employed 
sample member, we decomposed the survey-UI ratio of earnings per employee into two 
parts: the survey-UI ratio of earnings per employee for the subsample of individuals who 
were reported as employed in both data sets; and the survey-UI ratio of earnings per 
employee for the subsample of individuals who were reported as employed in only one 
of the data sets. 

The first step in this analysis was to examine the distribution of observations in 
which reported employment was consistent or inconsistent between the two data sourcesr . 
To do so, we created a "pooled" analysis sample in which each follow-up quarter for 
each member of the 12-month comparison sample was a separate observation. Thus, 
each sample member was represented by four observations (for quarters 1, 2, 3, and 4) 
in the pooled analysis sample. We refer to each of these observations as a person- 
quarter. 

Exhibit E.8 presents the percentage of these person-quarters for which neither data 
source or both data sources reported some earnings (and hence were consistent) and the 
percentage of these person-quarters for which either the First Follow-up Survey only or 
UI earnings data only reported some earnings (and hence the two sources were 
inconsistent). As shown in the last row of the exhibit, the two data sources report 
employment status consistently for 67 percent to 78 percent of the person-quarters in the 
pooled analysis sample. The remaining 22 percent to 33 percent of the person-quarters 
are divided roughly evenly between observations with employment reported only by the 
survey data and observations with employment reported only by the UI data. This 
pattern persists even if we look at each of the first, second, third, and fourth follow-up 
quarters separately (not shown in the exhibit). 

We then compared mean earnings per employee for person-quarters with 
employment reported in both data sources and for person-quarters with employment 
reported in only one data source. Exhibit E.9 presents ovv findings, which indicate that 
even when we compare earnings per employee for the subsamples with employment 
reported in both data sets, the survey-based estimates exceed the Ul-based estimates by 
a factor ranging from 24 percent in the case of adult women to 56 percent in (he case of 
male youths. The sources of these relatively higher survey-based estimates of earnings 
per employee might be second jobs or tips reported only in the survey data or over- 
reporting of earnings or job spell lengths in the survey data. 



41,; 



JTPA 18-MONTH IMPACTS / SURVEY-UI DATA COMPARISON • 353 



As shown in the second panel of Exhibit E.9, mean earnings per employee for 
person-quarters with employment reported only in the survey are more than twice as 
large as those for person-quarters with employment reported only in the'UI data. 
Moreover, the estimated earnings for person-quarters with employment reported only in 
the survey (and therefore "missed" by the UI data) are, on average, of nearly the same 
magnitude as the survey-based estimates for person-quarters with employment reported 
in both data sources (top panel).* In other words, the UI data may miss some quarters 
in which sample members had fairly typical earnings. A plausible explanation for this 
pattern is that UI data miss fairly typical quarters because of random errors in, say, 
reported social security numbers. 

Exhibit E.9 also indicates that estimated earnings from employment reported only 
in the UI data (and thus missed in the survey data) are, on average, about half as large 
as the Ul-based estimates of earnings from employment reported by both data sources. 
The survey data may therefore miss low-wage jobs, or person-quarters during which 
sample members were employed only briefly. This pattern may arise if survey 
respondents tended to forget short-term, low-wage jobs. It may also reflect misreporting 
by survey respondents of the start and end dates of their job spells, such that portions of 
quarters in which the respondents were actually employed went unrecorded. 



Differences in Estimated Program Impacts 
on Earnings and Employment 

Given the substantial diff'erences in earnings estimates from the two data sources, a 
crucial question is whether these diff'erences translate into diff'erent program impact 
estimates. As noted earlier, the answer depends on whether the survey-UI discrepancies 
differ substantially between the treatment group and the control group. 

Fortunately, this does not appear to be the case for any of the target groups. The 
ratio of mean survey earnings to mean UI earnings during the first four follow-up 
quarters was 1.35 for adult women in the treatment group and 1.35 for adult women in 
the control group. For adult men the rations were 1.53 and 1.52, respectively; for 
female youths they were 1.47 and 1.48; and for male youths, 1.79 and 1.83. Hence, 
neither data source appreciably over- or under-reported the average earnings of the 
treatment group or the control group relative to the other, although there was a slight 
reporting difference for male youths. 



8. Wc must qualify the notion thai ihc UI dau "miss" quarters wilh earnings, since " employ mcnl" 
reported only 'n the survey may arise from erroneous reporting of job spells in the survey. 
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Exhibit E.IO presents estimates of program impacts on quarterly and 12-month 
earnings, by target group, using both survey data and UI data. For all four target groups 
the differences between the survey-based impact estimates and the Ul-based estimates are 
not statistically significant (columns 3 and 6).^ 

Most of the difference between the impact estimates from the two data sources 
reflects a "scaling" of the result due to the fact that the survey reports more earnings than 
UI wage records for both the treatment group and the control group. For example, if the 
survey -UI earnings ratio were 1.5 for both the treatment and the control group the impact 
estimate from the survey data would be 1.5 times the corresponding estimate from the 
UI data. 

To eliminate this scaling effect when comparing the impact estimates from the two 
data sources, one can express the estimated impacts as a percentage of their respective 
mean control group earnings. Doing so for the estimates of impact on total earnings 
during the four follow-up quarters in Exhibit E.IO yields an 11.4 percent survey-based 
impact estimate and a 10.9 percent Ul-based impact estimate for adult women. 
Corresponding estimates are 4.1 percent and 3.5 percent for adult men; less than 0.1 
percent and less than 0.1 percent for female youths; and -8.2 and -6.5 percent for male 
youths. Hence, the impact findings from the two data sources tell virtually the same 
story for each target group, although they differ most noticeably for male youths. 

A key exception to this rule is the impact estimate for male youths with 2 previous 
arrest, the small minority of male youths who accounted for most of the negative average 
impact estimate for male youths overall (see Chapter 6). For male youths with a 
previous arrest, the survey-UI earnings ratio was 1.89 for treatment group members and 
2.62 for control group members. For this group, the survey data yielded a significant 
$-1,777 estimated earnings loss during the first four follow-up quarters, whereas UI data 
yielded an insignificant $89 estimated earnings gain. 



9. Wc icsicd whether the survey-based and Ul-^jascd estimates of impacts on earnings were significantly 
different by running a regression of the difference between survey-reported and Ul-rcportcd earnings on an 
indicator variable set equal to one for treatment group members and zero for control group members. The 
two impact estimates were significantly different if the coefficient of the treatment indicator was significantly 
different from zero. 

10. Slight differences between the survey-Ul earnings ratios for the treatment and control groups can 
generate a much larger proportional difference between the impact estimates from the two dau sources. 
Consider the following example: (1) mean Ul-bascd earnings are S5,500 for the treatment group and $5,000 
for the control group and (2) mean survey-based earnings are 1.70 times mean Ul-based earnings for the 
treatment group and 1.65 times mean Ul-bascd earnings for the control group. The estimated program 
impact is thus $500 according to the UI dau and $1,100 according to the survey dau. The survey-based 
impact estimate is therefore 2.2 times the UNbascd iftipa£L^*n^*^c. 
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Preliminary analyses of the types of jobs reponed on the survey (mostly low-wage 
service jobs) do not provide any reason to doubt the validity of the survey data. 
However, until we can fiilly explain the discrepancy between the impact estimates based 
on the survey data and those based on the UI data, one should interpret with caution the 
survey-based impact estimates for male youths with a previous arrest presented in this 
report. In our final report we will explore this issue in much greater detail. 

Program impact estimates from the two data sources were much more consistent 
with each other for male youths with ro previous arrest. They were $-185 from the 
survey data and $-277 from the UI data. Therefore, for most of the study sample-~-adult 
women, aduh men, female youths, and male youths with no previous arrests— the two 
data sources produced similar results. 



Summary 

The preceding findings indicate that average earnings of the 12-month comparison sample 
are substantially higher when estimated from First Follow-up Survey data than when 
estimated from UI earnings data. This differential was consistently observed for all four 
target groups in the study, for all 14 sites included in this analysis, and for all four 
calendar quarters after random assignment. Hence, the problem does not appear to 
rifiect idiosyncratic, localized issues but rather one or more pervasive differences 
between how the two data sources measure earnings. 

With the information currently available we have been able to explore the reasons 
for the survey-UI difference in earnings estimates to a limited extent. Our final report 
will present a more detailed analysis. Nevertheless, from the present analysis it appears 
that: 

• Almost all of the survey-UI difference in average earnings is due to higher 
survey-based estimates of earnings per employed sample member; very little 
of the difference is due to a difference in estimated employment rates 
between the two data sources. 

• Much of the survey-UI difference in average earnings is due to higher 
survey-based estimates of earnings for those sample members who were 
reported as employed by both data sources. 

• The remainder of the difference reflects the possibility that the survey tended 
to miss low-paying jobs, or jobs that sample members held only briefly, 
whereas the UI wage records tended to miss jobs with roughly average 
earnings per quarter. 
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• The extent to which differences in the earnings measures from the two data 
sources were translated into different estimates of program impacts' was 
limited. Estimates of program impacts on earnings were not statistically 
significantly different between the two data sources for any of the four target 
groups. For one target group, however— male out-of-school youths— the 
point estimates of program impacts were noticeably different. However, 
most of this difference was concentrated within the small minority of male 
youths with a previous arrest. Therefore on balance, the impact findings 
from the two data sources were consistent for adult women, adult men, 
female youths, and most male youths. 

G^ven the central role played by survey- and Ul-based earnings data in the National 
JTPA Study, and the considerable extent to which other researchers rely on both types 
of data sources, the analysis in this appendix must be viewed as a first step in the crucial 
task of exploring the nature and causes of the differences in the information obtained 
from the two data sources. 
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Exhibit E, I Distributions of Survey- cuui UI-Based Estimates of Mean U-Momh Earnings: 
Tlie lI^Month Comparison Sample, by Target Group 
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Sources: E«iimtlc« bued on Firil Follow-up Survey recpoiuet and eamingt daU from tUte Unemployment 
Insurance (UI) agencies. 

Notes: Usable UI earnings data for 2 of the 16 study sitea (Jersey City, N.J., and Marion. Ohio) are not yet 
available, and so these sites are excluded from this analysis. DaU on 4 out-of-school youths in Oakland, Calif, 
are included, although they were excluded from the 18-month analyse* elsewhere in this rq>ort. TesU of 
staiisiical significance were not performed for this exhibit. 
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Exhibit E, 2 Survey- and UI-Based Estimates of Mean 12-Month Earnings: The 12-Month 
Comparison Sample, by Target Group and Study Site 
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Sources: Eaimitcs based on First Follow-up Survey reiponses and earnings data from state Ul agencies. 
Notes: Usable UI earnings data for2 of ihe 16 Rudy sites (Jency City, N.J., and Marion, Ohio) arc not yet 
available, and so these sites arc excluded from this analysis. Data on 4 ooi-of-school youths in Oakland, Calif., 
arc included, although they were excluded from the IS-monih analyses elsewhere in this report. Tesu of 
statistical significance were not pcrforrod for this exhibit. Resulu are noi reported for sitss with fewer than 10 
sample numbers. 

a. UI earnings data for this site are currently incomplete. 



JTPA 18-MONTH IMPACTS / SURVEY-UI DATA COMPARISON • 359 



Exhibit E.3 Survey- and VI-Based Estimates of Mean 22'Month Earnings in Sites 

Located Near a State Border and in Those Not: The 12'Month Comparison 
Sample, by Target Group 

Adult Adult Female Male 

women men youths youths 

(Jl (2) (31 (4) 

Sites near a state border a 
First Follow-up Survey $ 5,715 $ 8,898 S 4,387 $ 7.110 
UI earnings data $4,352 $5,995 $3,003 $4,049 
Survey-UI ratio 1.31 1.48 1.46 1.76 
Sample size 2,989 2,226 1,207 929 

Sites not near a state border b 
First Follow-up Survey $ 4,655 $ 8,540 $ 2,884 $ 5,115 

UI earnings data $ 3,217 $ 5,303 $ 1,918 $ 2.641 

Survey-UI ratio 1.45 1.61 1.50 1.94 

Sample size 1,680 IJ03 732 518 

Sources: Esiimitcs based on Firsi Follow-up Survey reiponses and earnings dau from siaie UI agencies. 

Notes: Usable UI earnings dau for 2 of (he 16 rtudy sites Qenty City, N J., and Marion, Ohio) are not yet 

available, and so these sites are excluded from this analyns. Dau on 4 ou-of-school youths in Oakland. Calif.. 

are included, although they were excluded from the I8*monlh analyses elsewhere in this report. Tesu of 

statistical significance were not performed for this exhibit. 

a. This category includes the following aitea: Fort Wayne. Ind.; Cooaa Valley. Ga.; Providence. R.I.; 
Springfield. Mo.; Omaha. Neb.; Larimer County. Colo, and Northwest MinnesoU. 

b. This category includes Corpus Christi. Tex.; Jackson. Miss.; Oakland. Caiif.; Heartland, Fla.; Butte. Mort.; 
Decatur. 111. and Cedar Rapids, Iowa. 
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Exiiibit E.4 Survey- and UI-Based Estimates of Mean Quarterly Jid IZ-Momh Earnings: 
The 12'Month Comparison Sample, by Target Cro^p 





F'irst 


711 




First 












Ratio 


Follow-up 


earnings 


Ratio 




Survey 


data 


(1) / (2) 


Survey 


data 


(4) 1 (5) 


Period 


0) 


(2) 


(3) 




(5) 


(6) 






Adult wom^n 






Adult men 




Quarter I 


$ 1,090 


$ 789 


1.38*** 


$ 1,908 


$ 1.236 


1.54*** 


2 


1,294 


954 


1.36*** 


2,169 


1,432 


1.51*** 


3 


1,424 


1,075 


1.32*** 


2,311 


1.527 


1.51*** 


4 


1,526 


1,126 


1.36*** 


2,378 


1.545 


1.54*** 


All quarters 


5,334 


3,944 


1.35*** 


8,766 


5.740 


1.53*** 


Sample size 


4.669 


4.669 




3.529 


3,529 








Female youtlis 






Male youtlis 




Quarif^r 1 


$ 801 


$ 536 


1.49*** 


$ 1,365 


$794 


1.72*** 


2 


935 


618 


1.51*** 


1.583 


878 


1.80*** 


3 


1,028 


708 


1.45*** 


1,678 


917 


1.83*** 


4 


1,056 


732 


1.44*** 


1,771 


957 


1.85*** 


All quarters 


3,819 


2,594 


1.47*** 


6.396 


3.545 


1.80*** 


Sample size 


1.939 


1.939 




1.447 


1.447 





Sources: Estimates based on First FoUow-up Survey responses and earnings data from atate Ul agencies. 
Notes: Usable U2 earnings data for 2 of the 16 study sites (Jersey City, NJ., and Marion, Ohio) are not yet 
available, and so these sites are excluded from thia analysis. DaU on 4 out-of-school youths in Oakland, Calif., 
are included, although they were excluded from the IS-monih analyses elsewhere in this report. 
• Sutistically significantly different from one at the .10 level, at the .05 level, at the .01 level 



(two-uiled test). 
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ExJiibit E.5 Survey- and VI-Based Estimaxes of the Percentage Employed: 
Vie 12-Month Comparison Sample, by Target Croup 
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Noies: Usable UI earnings dau for 2 of the 16 study nies (Jei»cy City. N.J.. and Marion. Obio) aie not yet available. 



and so these sites are excluded from this analysis. DaU on 4 oui-of-school youths in Oakland, Calif., are included, 
although they were exclu<.1cd from the 18-month analyses elsewhere in this report. The -percenUge employed* is the 
percentage of the sample with any recorded earnings during the period. 

• Siiiiistically signiflcamly different from one at the .10 level, at the .05 level, at the .01 level (two-uilcd test). 
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Exhibit E. 6 UI-Based Estimates of the Pt rceruage Employed: The ll-Month 

Comparison Sample and the Subsample with Six Quarters of VI Earnings 
Data, by Target Group 
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Source: Estimates based on earnings data from state UI agencies. 

Notes: Usable UI earnings dau for 2 of the 16 «udy sites (Jersey City, N J., and Marion, Ohio) ape not yet 
available, and so these sites art excluded from this analysis. Dau on 4 oui-of-ichool youth* in Oakland. Calif., 
arc included, although they were excluded from the iS-nwnth analyses elsewhere in this report. The 
'percentage employed" is the peicenuge of the sample with any recorded earnings during the period. 
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ExhibU E. 7 Survey- and UI-Based Estimates of Mean Quarterly and 12-Month Earnings 
per Employed Sample Member: The 22'Monih Cotnparison Sample, by 
Target Group 
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3.307 


2.550 


1.30 


All quarters 


7,224 


5,212 


1.39 


10.645 


6.934 


1.54 


Sample size 


4,669 


4,669 




3,529 


3,529 








Female youths 






Male youths 




Quarter 1 


S 1,708 


$ 1,052 


1.62 


S 2.329 


S 1,289 


1.81 


2 


1,844 


1,174 


1.57 


2.440 


1,422 


1.72 


3 


1,991 


1,320 


1.51 


2.574 


1.492 


1.73 


4 


1.897 


1.404 


1.35 


2,524 


1,587 


1.59 


All quarters 


5,478 


3,457 


1.58 


7,890 


4.254 


1.85 


Sample size 


L939 


h939 




1.447 


1,447 





Source*: Estimate* based on Firxl Follow-up Survey retponsea and earnings dau from suie UI agenciea. 
Noies: Usable UI eamingi dau fori of the 16 «udy sites (Jeficy City. N.J.. and Marion. Ohio) arc not yet 
available, and so these sitea arc excluded from this analysts. Dau on 4 out-of-scfaool youths in Oakland. Calif., 
arc included, although they were excluded from the 18-montb analyaea elsewhere in this report. Teau of 
stjuistical significance were not performed for this exhibit. Mean e«raiags per employed sample member was 
computed by dividing mean earnings per sample member (Exhibit E.4) by the perccnuge of sample mcmbcn 
who were employed (Exhibit E.5). and multiplying this reault by l(X). The "perccnuge employed* is the 
percenuge with any recorded earnings during the period. 
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Exhibit E.8 Percentage of Person-Quarters in Which Employment Was Reported in the 

FirsJ Follow-up Survey Only, UI Earnings Data Only, Both Daia Sources, or 
Neither Source: The 12'Month Comparison Sample, by Target Group 



AduU Aduh Female Male 
women men youths youths 
Data source qi O)^ (S)^ (4) 

Inconsistent reporting 

First Follow-up Survey only 12.0% 16.4% 11.9% 18.2% 

Ul earnings dau only 10.3 10.8 13.0 14.8 

One source or the other 22^4 2^2 24^9 33.0 

Consistent reporting 

Neither source 31.1% 20.0% 35.8% 20.5% 

Both sources 46.6 52.8 39.3 46.5 

Neither or both sources 77.6 72.8 75.1 67.0 



Sources: Estimates based od First FoUow-up Survey responses and eanuDgs dau from state Ul agencies. 
Notes: Usable UI eanuDgs dau for 2 of the 16 study sites (Jeney City, NJ.. and MatioD, Ohio) are oot yet 
available, and so these sites are excluded from this analysis. Dau on 4 out-of-scfaool youths in Oakland, Calif., 
are included, although they were excluded from the 18- month analyses elsewhere in this tepott. TeAs of 
statistical significance were not perfonned for this exhibit. "Employment* is defined as any recorded earnings 
during the period. 
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Exhibit E.9 Mean 22-hionth Earnings for Person-Quarters with Employment Reported in 
One Data Source or Both Sources: The 12'Month Comparison Sample, by • 
Target Group 





Aduli 


Adult 


Female 


Male 


Data sources 


women 


men 


youths 


youths 


reporting employment 


(1) 


a) 


(3) 


(4) 


Both sources 










Suivey-base<i estimate 


S 2.375 


$ 3.216 


$ 1,972 


$ 2.527 


Ul-based estimate 


1.915 


2.462 


1,434 


1.620 


Survey-UI ratio 


1.24*** 


1.31*** 


1.38*** 


1.56*** 


Sample size 


8,699 


7,456 


3,050 


2,691 


Only one source 










Survey-based estimate 


S 1 .893 


$ 3.008 


$ 1.509 


$2,330 


Ul-based estimate 


907 


1.240 


649 


903 


Survey-UI ratio 


2.09*** 


2.43*** 


2,33*** 


2.58*** 


Sample size. Survey 


2,244 


2,314 


922 


1,054 


Sample size, UJ 


1,932 


1,529 


1,009 


855 



Sources: EAinuies based on First Follow-up Survey responses and earnings data from Aaie UI agencies. 
Noics; Usable Ul earnings dau for 2 of the 16 study sites (Jersey Cny, N.J.. and Marion, Ohio) are not yei 
available, and so these sites are excluded from this analysis. DaU on 4 out-of-KhooI youths in Oakland. Calif., 
arc included, although they were excluded from the IS-month analyses elsewhere in this report, •Enyloymcm" 
is defined as any recorded earnings during the period. 

• Statistically significantly different from one at the .10 level, at the .05 level, at the ,01 level (two-tailed test). 
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Exhibit E. JO Survey- and UI-Based Estimates of Program Impacts on Earnings: JTPA 
Assignees in the 12-Month Comparison Satnple, by Target Group 





Survey- 


UI- 


Difference. " 


Survey- 


UI- 


Difference, ^ 




based 


based 


in S 


based 


based 


in S 




impact 


impaa 


(1) - (2) 


impact 


impact 


(4) - (5) 


Period 


(I) 


(2) 


(3) 


w 


(5) 


(6) 






Adult women 






Adult men 




Quarter 1 


$ 91** 


$ 40 


$ 51 


$ 17 


$44 


S -27 


2 


163*** 


139*** 


24 


156* 


60 


96 


-a 


10 /^^^ 






1 io 
ilZ 


"711 
/o 


34 


4 


145** 


135*** 


10 


68 


12 


56 


All quarters 


566*** 


399*** 


168 


352 


194 


159 


Sample size 


4.669 


4.669 




3.529 


3.529 








Female youths 






Male youths 




Quarter 1 


$-14 


$-43 


$ 29 


$ -196** 


$-83 


S -112 


2 


-6 


-7 


1 


-28 


-39 


11 


3 


-47 


11 


-57 


-182* 


-50 


-132 


4 


73 


60 


13 


-149 


•68 


-81 


All quarters 


6 


21 


-15 


-555* 


-240 


-315 


Sample size 


1,939 


1.939 




1.447 


1.447 





Sources: Estimates based on First FoUow-up Survey responses and earnings data from ttatc Ul agencies. 

Notes: Usable UI earnings dau for 2 of the 16 study sites (Jereey City, N.J. , and Marion, Ohio) are not yet 

available, and so these sites are excluded from this analysis. Data on 4 out-of-school youths in Oakland, Calif., 

are included, although they were excluded from the 18-month aivUyscs elsewhere in this repo>-.. 

a. No difference in impacts is statistically significant at the . 10 level (two-tailed test). 

* Statistically significant at the . 10 level, ** at the .05 level, *** at the .01 level (two-tailed test). 




Appendix F 



Nonemollees and an Analysis of 
Their Participation in JTPA 



THIS appendix, which discusses the JTPA-related experience of the 36 percent of the 
treatment group that never enrolled in the program, serves two purposes. First, 
since nonenroUees did receive some JTPA services, it provides information on aspects 
of JTPA services received by the treatment group which would be missed by an exclusive 
reliance on enrollment data. Second, it provides needed context for interpreting the 
estimates of impact per enrollee presented in Chapters 4 through 7. Ideally, the report 
would present impacts per service recipient, but the only data available on the entire 
sample is JTPA enrollment. Information on JTPA services provided to nonenrolled 
assignees, therefore, provides information useful in assessing if impacts per enrollee 
differed in important ways from the ideal of impacts per service recipient. 

The appendix begins by describing the JTPA enrollment process and the incentives 
within JTPA to delay enrollment. It then describes a special study using a small sample 
which examined the extent to which assignees who were not enrolled received any JTPA 
services. 



Enrollment in JTPA Services 

Enrollment in JTPA occurs when SDA staff enter applicants' names and application data 
into tlie local JTPA manageme.nr. information system and assign them to one or more 
specific JTPA-funded activities. This step makes each of these applicants an official 
JTPA panicipant, wnose service receipt and progress is tracked and termination status 
(for example, employment and wages on leaving the program) is noted as part of the 
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JTPA performance standard system. By enrolling participants, SDA staff become 
accountable through the JTPA performance standard system for the costs JTPA incurs in 
serving them and their success when they leave the program. 

In this study, random assignment occurred after local staff had determined applicants 
were eligible for JTPA and had assessed their interests, skills, and service needs and 
recommended appropriate JTPA services. Once an applicant was designated a member 
of the treatment group through random assignment, local staff then tried to arrange 
classroom training in occupational skills, an on-the-job training position, basic education, 
or other possible JTPA services. 

Why would some members of the treatment group never be enrolled in JTPA? Four 
factors help explain why this would occur. 

1 . Applicants may change their minds about JTPA as they continue to seek 
other opportunities or learn more about the program. Many are looking for 
work on their own, and some will find employment. Some may discover 
different ways to finance the type of training they seek. And still others 
may decide that they are not interested in a job or training after all. 

2. The design of the JTPA program encourages local staff to make sure that 
applicants are going to participate in a service before they are enrolled and 
counted as a JTPA participant. An SDA's success in exceeding its 
performance standards is often given great weight as a sign of how well a 
program is operated and up to 6 percent of Title II-A funds are incentive 
grants based on SDA performance. The standards' visibility within the 
JTPA system goes well beyond their limited role in allocating incentive 
grants, however. This clearly encourages SDAs to focus on achieving their 
standards with respect to the various outcomes measured, but it also creates 
an incentive to be careful whom they enroll and to hold off on enrolling 
individuals (that is, having them count as part of the program) until they are 
placed in and begin a service that SDA staff feel is likely to produce 
successful impacts.^ In extreme cases, SDAs may delay enrollment in 
stand-alone job search assistance activities until individuals find employ- 
ment. 



1. Although the performance sundard syslcm has changed in recent years to include longcr-lerm 
measures of success (that is. thirteen weeks post-program) and those less closely tied to immediate 
employment and low costs, the basic incentives remain unchanged. SDAs can be seen as well run and gain 
some additional funding if a high proportion of people leaving their program find a job that pays well or if 
they attain a variety of employabilily-enhancing competencies. 
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3. Many SDAs believe that they have flexibility in defining the point at which 
individuals '^count" in their performance standards and responded to the 
incentives of the standards by delaying enrollment. In the initial years of 
JTPA, the Department of Labor adopted the position that JTPA was to be 
primarily controlled by states and localities. Therefore, the Department did 
not define precisely many key administrative terms, including the point at 
which enrollment should occur. In recent years, this federal stance has 
changed, but a holdover from the initial period is the continued practice of 
linking enrollment to the actual beginning of the intended service, be it the 
first day of class attended or the first workday for OJT. 

4. Despite the initial assessment that a client was appropriate for JTPA, staff 
may be unable to find a service provider willing to accept the person. 
Service providers often retain the final say on whether they will accept an 
applicant. Many classroom training agencies have entrance requirements or 
minimum initial skills and employers who could provide OJT will typically 
make their own assessment about whether they want the applicant as an 
employee. In addition, an applicant may be seeking classroom or other 
training at a time of the year when it is not offered. 

What Happened to NonenroUed Assignees? 

To determine this, the research team examined two questions. First, what was the typical 
policy concerning the point at which JTPA enrollment occurred? The finding mentioned 
earlier—that 64 percent of assignees were enrolled in JTPA— means that staff did not 
automatically enroll all members of the treatment group following random assignment. 
Through interviews with site staff, it was learned that most of the study sites enrolled 
individuals in classroom training when they attended their first class or in OJT when they 
worked their first day. In a few unusual sites, local staff could refer people to job search 
assistance or a job club without enrolling them and observe how they acted in this setting 
as part of an "extended assessment." The applicants referred in this manner to a job club 
might never be enrolled in JTPA unless they were to find a job or were referred to 
another activity because their behavior in the job club showed motivation and promise 
of employability. These findings established the possibility that staff could have worked 
with members of the treatment group following random assignment in an effort to arrange 
JTPA services without ever enrolling them. 

In order to understand the extent to which this occurred, the research team drew a 
random sample of treatment group members in 12 sites and interviewed local staff about 
their efforts to work with nonenrolled assij ees after random assignment. Twelve sites 
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were chosen from among the 14 still conducting random assignment at the time this 
special study was drawn. In the remaining two sites, logistical difficulties prevented us 
from conducting the study and processing the data. 

A ceiling of approximately 800 treatment group members was set for these sites to 
avoid overburdening site staff. Cases were apponioned among the sites based roughly 
on their proportion of treatment group members as of that date. In the case of the largest 
site (Fon Wayne, Indiana), the total sample was reduced from what it otherwise would 
have been because of concerns about staff burden. Cases were drawn during two time 
periods (November 1988-January 1989 and March 1989-June 1989) to lessen the 
influence of seasonal variation in local program practices. This might occur because the 
difficulty of arranging cenain types of services might vary over the year. Local staff 
received lists of treatment group members approximately three months after they were 
randomly assigned. This time period was chosen to allow sufficient time for JTPA 
enrollment to occur (most enrollments occurred within two months of random 
assignment) without delaying so long that local staff would have difficulty remembering 
whether and how they worked with assignees following random assignment. Local staff 
then identified those individuals on the list who were enrolled in JTPA; these were 
removed from the sample. This left 307 individuals who were nonenrollees. Local staff 
(usually SDA staff, but in some cases service provider staff as well) were then asked to 
report about post-random assignment contact with nonenrollees. To help staff in this 
task, the list of individuals included the date on which they were randomly assigned. 

Table F-1 presents the findings from this special study. The local staff had no 
contact with 15 percent of this sample after random assignment; basically, they were 
unable to locate them again. Another 11 percent of the sample reponed that they were 
no longer interested in JTPA, for a variety of reasons. Staff were able to recontact an 
additional 20 percent of the sample who were still interested in JTPA, but for whom the 
staff never did arrange any service. The remaining 53 percent of the sample of 
nonenrolled treatment group members had some post-random-assignment involvement 
with JTPA without being enrolled. The most common activity— provided for 36 percent 
of the nonenrollee sample— was one or more referrals to employers for a possible OJT 
position. Twenty percent of the sample participated in a job club or other job search 
assistance. This small study suggests that local staff did work with about half of the 
treatment group members who were never enrolled in JTPA, though in many cases little 
service was provided. 

Putting this together with the overall 64 percent enrollment rate in JTPA for the 
study sample, the results from this survey suggest that local staff worked (with a wide 
range of intensity and commitment to reading an enrollment) with slightly more than 80 
percent of the treatment group. This group consists of the 64 percent who were enrolled. 
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plus an additional 18 percent (that is, half of the 36 percent who were nonenrollees) who 
were never enrolled but did have some post-random assignment JTPA involvement. The 
remaining individuals could not be recontacted after random assignment, lost interest in 
the program (for a variety of reasons), or were interested in the program but never 
received any JTPA services. 

Ir. general, the extent of JTPA services to nonenrolled assignees varied by service 
strategy. It was highest for those in the OJT/JSA service strategy and lowest for those 
in classroom training. 

To summarize these findings, about half of the nonenrolled assignees in this sample 
received some type of JTPA services following random assignment. For those in the 
classroom training and OJT/JSA service strategy, ifiese services typically were much less 
intensive than those received by enrollees. In most cases, the nonenrolled JTPA services 
consisted primarily of some additional assessment and counseling and referrals to 
potential service providers or on-the-job training employers. However, those referred 
to job search assistance may have received a service similar to others who received job 
search assistance as a stand-alone activity but who did find employment and were 
enrolled in the program. In any case, the fact that the SDA staff did not enroll these 
individuals in JTPA suggests that these referrals and services did not lead directly to 
employment. 
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Exhibit FJ Distribution of Further JTPA Activity and 
Service Receipt after Random Assignment: 
A Random Sample of Treatment Group 
Nonenrollees in the 18-Month Study Sample 



Percentage 



of 

Action group 

No further contact 14.7 % 

Contact, but not eligible 1.3 

Client no longer interested 11.1 

Reasofis: 

Got job on own 4.6 

Moved 1 .6 

Health problems 1.3 

In another program 1 .0 

Uknown reason 2.6 
Client interested, made contact 

only; received no services 20.2 

Client interested; received some service 53.1 
Service: 

Received further assessment and 

counseling 11.1 
Referred to classroom training 

provider(s) 5.2 

Provided support service 2.3 
Referreo to employer(s) for 

possible OJT 36.2 
Participated in job club or 

job search assistance 19.9 



Source: Sample of 307 iretimeni group members who never enrolled in JTPA. 
Note: Sample members receiving some service could receive more than one 
lypc of serv ice. 



Appendix G 
Estimation of Impacts on Latent Wage Rates 



A program can affect earnings through either or both factors: the time spent in 
employment and the income people are able to earn per unit of time in employment. 
For those who work, hourly earnings are a measure of the latter concept.^ Unfortunately, 
we do not observe any outcome for nonworkers that is comparable to hourly earnings. 
We define the latent wage rate as the hourly wage rate that a person could command if 
she worked. For a person who works, the latent wage rate is equal to actual hourly 
earnings. For a nonworker, the latent wage rate is unobserved. This appendix repons 
the results of an attempt to estimate impacts on latent wage rates by use of a selection 
modeling procedure that relies on restrictive assumptions about the determinants of latent 
wage rates and employment. 



Model and Estimation Method 

We apply a two-step method due to Heckmj:, ii976). We make the following 
assumptions about the joint distribution of the latent wage rate, w,., and employment 
status, /n,, within the target group or subgroup: 



1 . Hourly earnings may also be a useful indicator of produciivily. Classical economic theory predicts 
that Ihc wage will equal the value of the worker's marginal product. Reasons why this equality need not hold 
include market imperfections that cause the wage to deviate from value marginal product; costs of fringe 
benefits and payroll uxes paid by the employer; costs of general training borne by the worker in the form 
of reduced wages; and returns to specific training shared between the worker and the employer. Becker 
(1975. Chapter II, Section 1) gave a theoretical analysis of the last two points. 
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1 (employed) if a'x. + u. > 0 
' 0 (not employed) if a'x. + u. ^ 0 

where x, is a vector of regressors consisting of a constant, a dummy variable for 
assignment to the treatment group, and a set of baseline characteristics; is normally 
distributed with zero mean and constant variance, conditional on x,.; the expectation of 
v„ conditional on x, and is zero; and the vector a, the vector /3, and the scalar 7 are 
unknown parameters. In this model, the element of /S that corresponds to the treatment 
group dummy variable is the impact of assignment to the treatment group on the latent 
wage rate. 

If the assumptions above are true, Heckman's (1976) two-step method yields a 
consistent estimate of /S: First, include both workers and nonworkers in the target group 
or subgroup in a probit regression of employment status on x,.. Then, include all workers 
in a linear regression of hourly earnings on x, and the inverse Mills ratio term 
^a'x,)/*(a'xp, where a is the estimated coefficient vector from the probit regression, 
<t> is the normal probability density function, and * is the cumulative normal distribution 
function. In this second-stage regression, the vector of estimated coefficients on x,- is the 
estimate of /S. 

Defining w,- to be the ratio of total earnings to total hours worked during the fifth 
and sixth quarters after random assignment and m,- to equal 1 if person i was employed 
at all in those quarters and 0 if not, we used this two-step method to estimate impacts on 
u'.. We estimated separate regressions for each target group and for each service strategy 
subgroup within each target group.' The regressors x. were those listed in Appendix D, 
Section 4 (omitting certain variables in the subgroup regressions to avoid 
multicollinearity). 

The estimated coefficient on treatment in the second-stage regression is our estimate 
of the impact of assignment to the treatment group on latent wage rates. We estimate the 
control mean by substituting the target group or subgroup mean covariate values into the 
estimated latent wage equation and setting the treatment group dummy variable and the 
inverse Mills ratio term to zero. 



2. Wc used dau on First Follow-up Sup/cy respondents only. Unlike he estimates of impaet on the 
earnings and employment of adult women, the estimates of impact on the lalei t wage rates of adult women 
arc not^'derived from'a nonrcsponscbias adjustment involving Unemployment h\s.irancc(U I) records, because 
the UI data do not measure hours worked. 
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Our significance tests use conventional estimates of standard errors from the second- 
stage regressions. Because these standard error estimates are not generally consistent, 
we also computed the White (1980) estimates, which are consistent (Amemiya 1985, pp. 
370, 387), for several of the wage regressions. The conventional and White standard 
error estimates did not differ appreciably. 

Note that because the employment and latent wage equations in our model contain 
the same regressors, the inverse Mills ratio term in the second-stage regression is simply 
a nonlinear transformation of the other regressors, including treatment. (It is difficult to 
make a convincing argument that a regressor in one equation neither influences the other 
outcome nor is correlated with omitted variables that influence the other outcome.) The 
model assumptions of linearity, normality, and homoskedasticity are essential to the 
reliability of the procedure. Goldberger (1983) and others have shown that under 
plausible departures from these assumptions, estimators of this type can have substantial 
bias. 



Results 

Exhibit G. 1 shows the estimated control mean and impact for each target group and for 
each service strategy subgroup within each target group. None of the estimated impacts 
are statistically significant at the .10 level. 

In light of the known fragility of the procedure under departures from the model 
assumptions, the results cannot be regarded as definitive. Although it is possible to test 
a subset of the model assumptions (e.g., Andrews 1988) or to apply methods that rely 
on less restrictive assumptions (e.g., Newey, Powell, and Walker 1990), no econometric 
procedure can provide definitive estimates of impacts on latent wage rates without 
information external to our data, given that the outcome is unobserved for the substantial 
fraction of the sample that was not employed. 
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Exhibit C. J fistimaied hnpacts on Lateni Hourly Wage Rales in the Fifth and Sixth Quarters after 
Random Assignment: JTPA Assignees in the IS-Monih Study Satnple, by Target Group 
and Service Strategy Subgroup 





Sample size 


Sample size 










(workers and 


(workers 


Conlrol 


Impact, 




Target group/ 


nonworkers) 


only) 


tnean 


inS 


Standard error 


subgroup 


(1) 


a) 


(S) 


(4) 


(S) 


Adult women 








) v. 14 


A 1 A 


Classroom training 




l,C>Oo 




A 1 C 


A 1 *7 
U. 1 / 












All 


Other services 


1,147 


785 


5.82 


-0.05 


0. 18 


Adult men 


4,419 


3.374 


6.68 


0.00 


0.11 


Classroom training 


1,057 


807 


6.88 


-0.12 


0.23 


OJT/JSA 


2.250 


1.750 


5.72 


0.15 


0.15 


Other services 


1,112 


817 


6.67 


-0.17 


0.22 


Fenwie youths 


2,300 


1,509 


4.91 


0.05 


O.ll 


Classroom training 


1.045 


666 


4.79 


0.19 


0.16 


OJT/JSA 


545 


397 


5.29 


-0.15 


0.32 


Other services 


710 


446 


4.74 


0.04 


0.19 


Male youtlL« 


1,748 


1,420 


5.74 


-0.01 


0.14 


Classroom training 


526 


439 


5.56 


0.03 


0.21 


OJT/JSA 


615 


525 


5.92 


0.09 


0.29 


Other services 


607 


456 


5.72 


-0.19 


0.22 



Source: Estimaies based or. First Follow-up Survey responses. 

Note: None of the estimated impacts (column 4) were staiistically significant at the .10 level. 



Appendix H 



Supplementary Exhibits on 
Out-of-School Youth 
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Exhibit H.l Percentage Enrolled in JTPA Monthly: Female Youth Treatment .Group 

100% 



< 

f- 



80 S 



•g 60% 



c 

g) 40% 



20% 



0% 



6 9 12 

Months after random assignment 



15 



18 



Source: Unadjusted frequencies based on enroiimeni and iracicing dau from ihe 16 service delivery areas (SDAs). 
Noie: SanipJe size, ireainnent group = 1,586. 



Exhibit H.2 Impacts on the Distribution of Total 18-Month Earnings: Female Youth 
JTPA Assignees 







Control 


Difference, 




Assignees 


group 


in % pis. 


18-month earnings 


0) 


(2) 


(3) 


SO 


18.9% 


22.4% 


-25%** 


$1 - $2^37 


21.5 


19.5 


2.0 


$2^38 - 56^93 


21.9 


19.3 


2.6 


$6^94 - $11,762 


18.7 


19.3 


-0.6 


> $11,762 


19.0 


19i 


-0.5 



Chi-squarcd test of impact on entire distribution 



not significant 



Source: Estimates based on Fim Foliow«up Survey rcsponses. 

Notes: Sample size, assignees =1,586; control group =714. For the estimation procedure, sec 
Appendix D. 

• Statistically significant at the .10 level, at the .05 level, *** at the .01 level (chi-squarcd test or 
iwo-uilcd t-tesi). 



ERIC 



JTPA 1 8-MONTH IMPACTS / OUT-OF-SCHOOL YOUTH EXHIBITS • 379 



Exhibit H.3 



Impacts on the Percentage Employed and on the Mean Number ofVk'eeks 
and Hours Worked: Female Youth JTPA Assignees and EnroUees. 



Impact per assignee 





Coniroi 


In % pis. , 
weeks, or 




Inferred impact per 
enrollee, in % pis. , 


Period 


(h 


hours As % of (1) 

a) (3) 


weeks, or iiours 






Percentage employed 




Quarter 1 


44. 17c 


-0.6% 


-1.2% 


-0.9% 






-2.3 


-4.5 


-3.8 


3 


54.2 


-0.5 


-1.0 


-0.9 


4 


55.3 


0.8 


1.4 


1 .3 


5 


57.0 


0.1 


0.1 


0.1 


6 


59.3 


-1.0 


-1.7 


-1.7 


Anylirne during 
quarters 1 - 6 


78.1 


2.8 


3.6 


4.5 






Weeks worked 






Quarter 1 


4.8 


-0.4* 


-8.5% 


-0.7 


2 


5.5 


-0.6** 


-10.4 


-1.0 


3 


5.9 


-0.4 


-6.1 


-0.6 


4 


5.9 


0.0 


0.4 


0.0 


5 


6.1 


0.0 


0.6 


0.1 


6 


6.4 


-0.3 


-4.8 


-0.5 


All quarters 


34.7 


-1.6 


-4.6 


-2.6 






Hours worked 






Quarter 1 


174 


-16* 


-9.0% 


-25 




206 


-19* 


-9.0 


-30 


3 


230 


-18* 


-8.0 


-30 


4 


223 


1 


0.4 


1 


5 


229 


4 


1.6 


6 


6 


239 


-13 


-5.4 


-21 


All quarters 


1.302 


-61 


-4.7 


-99 



Source: Eslimwcs based on Finl Follow-up Survey responses. 
Noics: Sample size, assignees = 1.586'. control group = 7U. Ettimatet are regression-adjusled to oourol 
for diffcreoces in baseline chancteri sties between the treatment group and control gixxip; sec Appeodix D. 
* Suiistically significant ai the .10 level, *♦ at the .05 level. *** ai the .01 level (two-uil«d ic»t). Sigoiftcance 

levels for column 3 are identical to thoac in column 2. Teau of tmisticai significance were not performed 

for column 4. 
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Exhibit H,4 Percentage Impacts on Earnings and Its Components: Female Youth 



JTFA Assignees 



Period 


Earnings 
per assignee 

0) 


Workers 

per 
assignee 

(2) 


Weeks 
worked 
per worker 
(3) 


Hours worked 
per week 
worked 
(4) 


Earnings 
per hour 
ys'orked 

(5) 


Quarter 1 


-6.4% 


-1.2% 


-7.4% 


-0.7% 


3.0% 


2 


-5.9 


-4.5 


-6.2 


1,5 


3.4 


3 


-6.8 


-1,0 


-5.2 


-2,0 


1.3 


4 


0.2 


1.4 


-0.9 


-0.1 


-0.2 


5 


4.4 


0.1 


0.3 


1.1 


2.8 


6 


-4.5 


-1.7 


-3.0 


-0.7 


1.0 


All quarters 


-2.9 


3.6 


-7,9 


-0,1 


1.9 



Source: Esvimaics based on Firsl FoIIow-up Survey retpoiues. 

Notes: Sample size, assignee* = 1,586; conurol group « 7U. Ejiimne* are regression-adjusied lo control for 
differences in baseline characicrittics between the treatment group and control gnxip; »cc Appendix D- 
Columns 2 through 5 display the impact as a percentage <)f the co<Te«pooding control mean (not shown). For 
column 2 this means the impact on the employment rate i» displayed as a percentage of the mecn rate for the 
con;roI group. Tests of statistical significance were not perfonned for any of the columns in this exhibit. 
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Exhibit H,6 Impacts on the Percentage Employed and on the Mean Number of Weeks 
and Hours Worked: Female Youth JTPA Assignees, by Service Strategy 
Subgroup 



Perce ntage emplo yed Weeks wttrked Hours wvrked 

Control Impact, Control Impact, Control Impact, 
mean in % pts, mean in weeks mean in hours 
Period Ql (41 (S^ (6> 

Classroom training subgroup 



Quarter 1 


44.2 


-8.7%*** 


4.6 




164 


-46**» 




50.9 


-6.6** 


5.3 


-1.1*** 


197 


-44*** 


3 


49.9 


1.4 


5.7 


-0.5 


221 


-30** 


4 


49.9 


4.1 


5.4 


0.3 


206 


1 


5 


54.8 


0.9 


5.8 


0.2 


214 


8 


6 


57.1 


0.7 


6.2 


-0.1 


228 


-8 


AH quarters 


^ 75.2 


3.5 


32.8 


-2.4 


1,231 


-119* 








OJT/JSA subgroup 






Quarter 1 


53.9 


6.07c 


5.9 


0.3 


228 


16 


2 


55.1 


5.7 


6.2 


0.3 


238 


21 


3 


61.4 


1.4 


6.7 


0.0 


263 


6 


4 


64.1 


-1.2 


7.1 


-0.5 


275 


-7 


5 


67.0 


-3.5 


7.3 


-0.3 


282 


.2 


6 


70.1 


-6.3 


8.1 


-1.3** 


308 


-47** 


All quarters 


84.8 


3.6 


41.3 


-1.4 


1,593 


-13 








Other services subgroup 






Quarter 1 


37.4 


7.2%* 


4.0 


0.4 


142 


11 




50.3 


-2.4 


5.3 


-0.5 


194 


-11 


3 


54.3 


-3.7 


5.7 


-0.4 


215 


-18 


4 


55.5 


-1.0 


5.7 


0.1 


209 


6 


5 


53.4 


0.6 


5.8 


0.0 


213 


1 


6 


54.1 


0.6 


5.7 


0.1 


201 


7 


All quarters 


^ 76.2 


2.6 


32.1 


-0.4 


1,175 


-4 


Source: Estimates based on Fint Follow-up Survey responses. 

Notes: Sample size, classroom tnining subgroup: assignees = 704, control group = 341; OJT/JSA subgroup, 
assignees = 3SI, control group ^ 164; other seavioea subgroup: assignees = 501, control group = 209. Estimates 
arc regression-adjusted to control for differences in baseline charvctenstics between the ireatmcnl group and 
control group; see Appendix D. 

«. For columns 1 and 2 ('percenlage employed*) this row shows the perceniage of control group membecY who 
reported being employed at any lime during the follow-up pKriod and the estimated intact on this percentage for 
assignees, respectively. 

* Sutistically significant at the .10 level, ** at the .05 level, *** at the .01 level (two-uiled lest). 



JTPA 18-MONTH IMPACTS / OUT-OF-SCHOOL YOUTH EXHIBITS • 383 



Exhibit H, 7 Percentage Enrolled in JTPA Monthly: Male Youth Treatment Group 



807c 

< 




0% I ^ ^ ^ ^ ^ ^ ^ . . ^ ^ . 

0 3 6 9 12 15 18 

Months after random assignment 



Source: Unadjusted frequencies based on enroUment and tracking dau 
from the 16 service delivery areas (SDAs). 
Note: Sample size, treatment group 1,196. 



Exhibit H.8 Impacts on the Distribution of Total 18-Month Earnings: Male Youth 
JTPA Assignees 







Control 


Difference, 




Assignees 


group 


in % pis. 


18-month earnings 


(1) 


<2) 


(3) 


SO 


9.5% 


10.9% 


-1.4% 


SI - S5,236 


27.1 


22.3 


4.8** 


S5.237 - S10.210 


21.1 


22 3 


-1.2 


S10,211 - S17.222 


24.2 


22 3 


1.9 


> S17.222 


18.1 


22 3 


-4.2** 


Chi-squarcd test of impact on entire distribution 




not significant 



Source: Eaimaics based on First Follow-up Surve>' responses. 

Notes: Sample size, as$igne«$ » 1,196; control group «= 552. For the estimation procedure, see 
Appendix D. 

• SuiisiicaHy fignificant at the .10 level, at the .05 level, at the .01 level (chi-squared test or 
two-tiilod t-test). 
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Exhibit HS Impacts on the Percentage Employed and on the Mean Number of 
and Hours Worked: Male Youth JTPA Assignees and Enrollees 



Impact per assignee 







In % pis. , 




Inferred impact per 




Control 


v/eeks, or 




^nrnllee in Dls 




mean 


hours As % ofO) 


weeks, or iwurs 


Period 




(2) 


(3) 


(^) 






Percentage employed 




Quarter 1 


58.1% 


-3.4% 


-5.8% 






63.1 


-0.5 


-0.8 


-0.9 


J 


oo.o 


-2.1 


-3.0 


-3.3 


4 


71 'K 
/ 1. J 


-3.3 


-4.6 


-5.3 


5 


73.6 


-1.6 


-2.2 


-2.7 


o 


77 5 
/ /.J 


-2.7 


-3.6 


-4.4 


Anytime during 






1.7 




Cju^rters 1-6 


89.0 


1.5 


2.42.4 






Weeks worked 








6.4 


-0.6** 


-9.4% 


-1.0 


2 


7.0 


-0.2 


-3.1 


-0.4 


3 


7.7 


-0.5 


-5.9 


-0.7 


4 


7.9 


-0.4 


-5.1 


-0.7 


5 


8.2 


-0.1 


-1.7 


-0.2 


6 


8.5 


-0.4 


-5.1 


-0.7 


All quarters 


45.7 


-2 3* 


-4.9 


-3.6 






Hours worked 






Quarter 1 


257 


-30** 


-11.8% 


-48 


2 


287 


-9 


-3.3 


-15 


3 


326 


-27** 


-8.3 


-43 


4 


333 


-24* 


-7.2 


-38 


5 


343 


-14 


-4.0 


-22 


6 


346 


-24* 


-7.0 


-39 


All quarters 


K892 


-129** 


-6.8 


-205 



Source: Eslimaics based on Firsi Follow-up Survey responses. 

Notes: Sample size, tssignoes « 1,196; conirol grogp = 552. Ettimalcs »re regrcssion-»djuslod lo conirol 
for diffcrcnccf in baseline charweriflics between ihe ircaimcoi group and corurol group; tec Appendix D. 
• Suiislically significani al the . 10 level, *• al ihe .05 level, at ihe .01 level (iwo-iailod test). Significance 
Icveli for column 3 arc idenlical lo lho»e in column 2. Tens of staiistical significance were not perfonned 



for column 4. 
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Exhibit H.IO Percentage Impacts on Earnings and Its Components: Male Youth 
JTPA Assignees 







Workers 


Weeks 


Hours worked 


Earnings 




Earnings 


per 


Mforked 


per week 


per hour 




per assignee 


assignee 


per worker 


worked 


worked 


Period 


a) 


aj 


(3J 




(S) 


Quarter 1 


-14.1% 


-5.8% 


-4.0% 


-2.6% 


-2.6% 




-4.5 


-0.8 




-0.2 


-1.2 


3 


-8.4 


-3.0 


-2.9 


-2.6 


-0.1 


4 


-7.3 


-4.6 


-0.4 


-2.3 


-0.1 


5 


-5.3 




0.6 


-2.3 


-1.3 


6 


-9.2 


-3.6 


-1.5 


-2.0 


-2.3 


All quarters 


-7.9 


1.7 


-6.5 


-2.0 


-1.2 



Source: EslimAlci based on Fim FoUow-up Survey retponses. 

Notes: Sample size, assignees « 1 ,196; comrol jroup « 552. Esiimatcs arc rcgrcssion-adju led lo comrol for 
difference* in baseline chamclcrislics between ihe ireatmeal group and control group; sec AppcwJix D. 
Columns 2 through 5 display the impact as a percentage of the corrcspoKling control mean (not shown). For 
column 2 this means the impact on the employ racai rale is displayed as a perceniage of the mean r*:e for the 
conirol group. Tests of siaiisiical significance were noc performed for any of the columns in this exhibit. 
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Exhibii H.I 2 Jmpaas on the Percentage Employed and on the Mean Number of Weeks 
and Hours V/orked: Male Youth JTPA Assignees, by Service Strategy- 
Subgroup 





Percentage employed 


y<feeks worked 


Hours worked 




Control 


Impact, 


Control 


Impact, 


Control 


bnoaci. 




mean 


in % pts. 


mean 


in weeks 


mean 


in hours 


Period 


(1) 


(2) 


(S) 


(4) 


(5) 


(61 








Classroom training subgroup 






Quarter 1 


54.8 


-6.3% 


6.1 




228 


-54*» 


2 


58.2 


3.6 


6.4 


0.1 


243 


13 


3 


DO. J 




7.3 


0.1 


293 


3 


4 


68.8 


0.3 


7.7 


0.0 


309 


n 


5 




-U.-i 


8.0 


0.0 


326 


-9 


6 




1 fi 


8.3 


-0.2 


327 


-7 


All /iiiort^rc ^ 


89.1 


2.4 


43.8 


-1.3 


1,727 


-50 








OJT/JSA subgroup 






Quarter 1 


63.0 


1.5% 


6.8 


0.3 


289 


6 




71.8 


-4.0 


8.1 


-0.5 


347 


-24 


3 


76.8 


-5.0 


8.8 


-0.8 


391 


-53** 


4 


77.0 


-5.5 


8.4 


-C.3 


311 


-36 


5 


79.0 


-2.4 


9.1 


-0.3 


394 


-32 


6 


83.1 


-4.4 


9.3 


-0.7 


393 


-44* 


All quarters ^ 


92.5 


0.9 


50.4 


-2.3 


2,190 


-182 








Other serxnces subgroup 






Quarter 1 


57.5 


-7.7%* 


6.2 


-1.0* 


254 




2 


58.1 


-0.2 


6.4 


-0.2 


267 


-18 


3 


62.1 


-2.1 


7.0 


-0.6 


293 


-34 


4 


67.8 


-4.4 


7.6 


-0.9* 


315 


-43* 


5 


69.4 


-2.4 


7.4 


0.1 


305 


3 


6 


69.9 


-0.3 


7.9 


-0.5 


318 


-25 


All quarters ^ 


85.2 


1.6 


42.4 


-3.0 


1,752 


-169 



Source: Estimaics based on First Folio w-up Survey responses. 

Noics: Sample size, elassroo:: i ruining lubgnxip: assignees = 354. conirol group = 172; OJT/JSA lubgroup. 
assignees = 41 1, control group = 204; other services subgroup: assignees = 431. control group = 176. Esiimaic* 
arc rcgrcision-adjustcxi to control for differences in baseltne charmcicriiiics between the treMmem grxxjp and 
control group; see Appcwiix D. 

«. For columns 1 and 2 ("pcrccniage employed") this row shows the percentage of control group memben who 
reported being employed at any time dunng the follow-up penod and the estimated impact on this pcrxxntage 
for assignees. reipcciivel> . 

• Siaiistically significant at the .10 level. at the .05 level. at the .01 level (two-tailed test). 
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ExhibU 'H.13 Percentage Enrolled in JTPA Monthly: Treatment Group, All Youths 

100% r 



< 80% 




i 

0% ^ ^ ^ ^ ^ ' ' — ' ' ' ' — ' ' ' ' ' ' ' 

0 3 6 9 12 15 18 

Months after random assignment 



Source: Uiudjusicd frequencies btsed on enrollment and tracking dau from ihc 16 service delivery areas (SDAs). 
Note: Sample size, treatment group 2,782. 



ExJubirH.14 


Impacts on Earnings: All Youth JTPA Assignees and Enrollees 






Control 


Impact per assignee 


Inferred impact per 




mean 


In$ 


As % of (I) 


enrollee, in $ 


Period 


(I) 


a) 


(V 


W 


Quarter 1 


S 1,058 


$-113*** 


-10.7% 


S -182 


2 


1,232 


-57 


-4.6 


-91 


3 


1,399 


.97** 


-6.9 


-155 


4 


1,430 


>48 


-3.4 


-77 


5 


1.501 


-8 


-0.5 


-13 


6 


1,587 


-104** 


-6.5 


-166 


All quarters 


8.207 


-427* 


-5.2 


-684 



Source: Estimate* based on Fim Follow-up Survey reiponscs. 

Notes: Sample »i2£. assignees « 2,782; control group = 1,266. Esti male* are Pcgpciiion-adjusiod to control 
for differences in baseline characteristics between ihe treatment group and control group; lec Appendix D. 
♦ Statistically significanl at the .10 level, at the .05 level, at the .01 level (two-uilod lest). 

Significance levels for column 3 arc identical to those in column 2. Tests of st«;i$tical significance were nol 



pcrformod for column 4. 



4^2 
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Exhibii H.15 Impacts on the Distribution of Total 18-Month Earnings: All Youth 
JTPA Assignees 









Control 


Difference, 






Assignees 


group 


in 9c pts. 


IS-month earnings 


(1) 


(2) 


(3) 




$0 


14.8% 


17.4St 


-2.6%** 


$1 


- S3,598 


24.1 


20.7 


3.4** 


$3,599 


- $8,248 


22.3 


20.6 


1.7 


$8,249 


- $14,178 


19.8 


20.6 


-0.8 




> $14,178 


18.9 


20.7 


-1.8 



Chi-squared test of impacl on entire distribution 



Source: Estimates based on Firsi Follow-up Survey responses. 

Notes: Sample siztf, assignees = 2,782; control group 1,266. For the estimation procedure, sec 
Appendix D. 

• Siaiistically significani ai the .10 level, at the .05 level, at the .01 level (chi-squarcd test or 
two-iailed t-test). 
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Exhibit H. 16 Impacts on the Percentage Employed and on the Mean Number of 
Weeks and Hours Worked: All Youth JTPA Assignees and Enrollees 







Impact per assignee 








Jn % pis, . 




Inferred unpaci per 




Control 


weeks, or 




enrollee, in % pis. , 




mean 


hours As % of (J) 


weeks, or hours 


Period 


(1) 


a) 


(S) 


(4) 






Percentage employed 




Quarter 1 


50A% 


-1,6% 


-3.1% 


-2,5% 


2 


56.4 


-1.2 


-2 2 


-2.0 


3 


60.3 


-0.9 


-1.5 


-1.5 


4 


61.9 


-0.5 


-0.9 


-0,9 


5 


64.0 


-0.4 


-0.7 


-0.7 


6 


67,0 


-1.5 


-2.3 


-2,5 


Anytime during 








3.9 


quarters 1-6 


82.7 


04** 


2,9 






Weeks worked 






Quarter 1 


5.4 


-0,5** 


-8.6% 


-0.8 


2 


6.2 


-0,4** 


-6.3 


-0.6 


3 


6.7 


-0.4* 


-5.4 


-0,6 


4 


6.7 


-0.1 


-1.5 




5 


7-0 


0.0 


0.0 


0.0 


6 


7.3 


-0.3* 


.4.4 


-0.5 


All quarters 


39.2 


-1.6* 


-4.2 


-2.7 






Hours worked 






Quarter 1 


209 


-21*** 


-10.1% 


-34 


2 


240 


-13 


.5.4 


-21 


3 


270 


-20** 


-7.5 


-33 


4 


270 


-8 


-3.1 


-14 


5 


277 


.2 


-0.7 


-3 


6 


284 


-15* 


-5.5 


-25 


All quarters 


1,550 


-80** 


-5-2 


-131 



Source: Estimaics based on Firel FoUow-up Survey reaponset. 

Notes: Sample size, assignees = 2,782; control group = 1,266, Eitimaie* are rcgrcfsion-adjusied lo control 
for difference* in baseline characicrisiics between the tttatinc&t group and control group; ace Appendix D. 
♦ Sutistically tignificanl at the .10 level. at the .05 level, at the .01 level (two-iailed icit). Significance 
levels for column 3 arc idcnticaJ to those in column 2. Tc«u of tuiittical significance were not performed 



for column 4. 
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ExJiibix H,17 Percentage Impacts on Earnings and Its Components: All Youih JTPA 
Assignee. 









V/orkers 


Weeks 


Hours worked 


Earnings 


Period 




Earnings 


per 


worked 


per week 


per hour 




per assignee 


assignee 


per worker 


worked 


worked 






(1) 


(2) 


(S) 




(5) 


Quarter 


1 


-10.7% 


-3.1% 


-5.5% 


-1.7% 


-0.7% 




2 


-4.6 


-2.2 


-4.1 


1.1 


0.8 




3 


-6.9 


-1.5 


-4.0 


-2.2 


0.6 




4 


-3.4 


-0.9 


-0.8 


-1.5 


-0.3 




5 


-0.5 


-0.7 


0.5 


-0.6 


0.2 




6 


-6.5 


-2.3 


-2.1 


-1.1 


-1.2 


All quarters 


-5.2 


2.9 


-6.9 


-1.1 


0.0 



Source: Estinuies based on Firrt FoUow-up Sucv^ rMpoiues. 

Nolc»: Sample »ize. atsigneec = 2.782; conirol group = 1.266. Esiiouiet arc rcgrcasion-adjuslcd lo control for 
differences id baseline chaiadcrislics between ihe treaimeoi group and conirol group; see Appendix D. 
Columns 2 ihrough 5 display ihe impaa as a perccnugc of the corresponding conirol mean (nd thown). For 
column 2 this means the impaa on the employment rale is displayed as a percenuge of the mean nte for the 
control group. Tests of statistical significance were not performed for any of the columns in this exhibit. 



4^5 



392 • JTPA 18-MONTH IMPACTS / OUT-OF-SCHOOL YOUTH EXHIBITS 



Exhibit H.18 Impacts on Anainmem of a Training^Related High School Diploma or 
CED Certificate: All Youth JTFA Assignees Overall and High School 
Dropout Subgroup, by Service Strategy Subgroup 



Percentage attaining a training- related 
high school credential ^ 





Sample 

a 

Size 

(1) 


Control 

Assignees group 
(2) (3) 


Difference, 
in pts. 
(4} 






Classroom training subgroup 




Full sample 




15,8% 8,5% 


7.3%*** 


High school dropouts 


769 


30.7 17,2 


13. 5*** 






OJT/JSA subgroup 




Full sample 


Mil 


4,7 1.9 




High school dropouts 


408 


12.7 5.3 


7.4*** 






Other services subgroup 




Full sample 


1.267 


18.8 12,5 


6.4*** 


High school dropouts 


828 


28.8 19.0 


9.8*** 






All subgroups 




Full sample 


3,889 


13,7 7.8 


5.9*** 


High school dropouts 


2,005 


26.3 15.4 


10.9*** 



Sources: Unadjusted frequencies based od Background la formation Fom responses and First Follow-up 

Survey responses, 

a Treatment and control groups combined. 

b. 'Atiainment of a training-related high school credential* is defined as the combination of having received 
some school or training service and having attained a high school diploma or General Educational 
Development certificate at some time during the 18-month foUow-up period. 

• Suiistically significant at the .10 level. ai the .05 level. at the .01 level (two-tailed test). 
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Exhibil H.J 9 Impacts on Earnings: All Youth JTPA Assignees and Enrollees, 
by Service Strategy Subgroup 

Control Impact per assignee Inferred impact per 

f^'^n InS As% of (I J enrollee, in $ 

^^r^o^ (21 (2) (3) (4) 



Classroom training subgroup 



Quarter 1 


S 


916 


S -250*** 


-213% 


s 


-366 


2 




1,063 


-92 


-8.6 




-135 


3 




1,262 


-98 


-7.7 




-143 


4 




1,267 


10 


0.8 




15 


5 




1,339 


31 


2.3 




45 


6 




1,434 


-44 


-3.1 




-64 


All quarters 




7,281 


>443 


-6.1 




-648 








OJT/JSA subgroup 






Quarter 1 


S 


1,351 


S 39 


2.8% 


S 


72 


2 




1.566 


-18 


-1.1 




-33 


3 




1,750 


-102 


-5.8 




-188 


4 




1,778 


-93 


-5.2 




-171 


5 




1,869 


-39 


-2.1 




-72 


6 




2,019 


-222** 


-11.0 




-408 


All quarters 




10,333 


-435 


-4.2 




-800 








Other services subgroup 






Quarter 1 


$ 


992 


$ -107 


-10.7% 


s 


-172 


2 




1,166 


-84 


-7.2 




-135 


3 




1,282 


-129 


-10.1 




-208 


4 




1.375 


-149* 


-10.9 




-240 


5 




1,414 


-77 


-5,4 




-124 


6 




1,423 


-104 


-7,3 




-168 


All quarters 




7,652 


-650 


-8.5 




-1,048 



Source: Esii males based on Frrsi Follow- up Survey responses. 
Noies: Sample size, classroom training subgroup: assignees = 1,058. comrol group =513; OJT/JSA 
subgroup: assignees = 792, comrol group = 368; oihcr services subgrr-jp: assignees = 932, comrol group 
= 385, Eaimaies arc regressioiv- adjusted lo comrol for differences in baseline characteriflics betweea the 
ueaimem group and comrol group; see Appendix D. 

• Suiisiically significam at the . 10 level. at the .05 level, ••• at the .01 level (two uilcd lest). 
Significance levels for column 3 arc idemical to those in column 2. Tests of suiislicai significance were 
not performed for column 4. 
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Exhibit H.20 Impacts on the Percentage Employed and on the Mean Number of Weeks, 
and Hours Worked: All Youth JTPA Assignees, by Service Strategy 
Subgroup 

Percentage employed Weeks worked Hours worked 
Control Impact, Control Impact, Control Impaci. 
mean in % pts, mean in weeks mean in }iour:> 
Period (J) (2) (3) (4) (5) (6) 

Classroom training subgroup 



Quarter 1 


47.9 


-8,0%*** 


5.1 


.2 2*** 


186 


.49»»* 


: 


53.1 


-2.8 


5.7 


-0.7** 


212 


-24** 




55.7 


1.3 


6.2 


-0.3 


246 


-20 


4 


56.4 


2.6 


6.2 


0.1 


243 




5 


60.4 


0.8 


6.5 


0.1 


252 


1 


6 


64.2 


-0.1 


6.9 


-0.1 


261 


-7 


AU quarter^ 


79.8 


3.2 


36.6 


-2.1 


1,400 


-102* 








OJT/JSA subgroup 






Quarter 1 


58.8 


3.7% 


6.4 


0.3 


261 


10 


2 


64.4 


-0.1 


7.2 


-0.1 


296 




3 


69.7 


-2.1 


7.8 


-0.4 


330 


-23 


4 


70.8 


-3.2 


7.7 


-0.3 


326 


-IS 


5 


72.7 


-1.9 


8.2 


-0.2 


338 


-13 


6 


76.7 


-4.9* 


8.6 


-0.8** 


349 


-39** 


All quarters^ 


89.0 


2 i 


45.9 


-1.5 


1,900 


-86 








Other services subgroup 






Quarter 1 


46.4 


0.1% 


5.0 


-0.2 


193 


-17 


2 


53.7 


-1.1 


5.8 


-0.3 


226 


-12 


3 


57.8 


-2.8 


6.2 


-0.4 


248 


-'^'^ 


4 


61.2 


-2.6 


6.6 


-0.4 


258 


-16 


5 


61.2 


-1.5 


6.6 


-0.1 


257 


0 


6 


61.7 


-0.2 


6.7 


-0.2 


256 


-9 


All quarters^ 


80.4 


2.0 


36.9 


-1.6 


1,437 


-75 



Source: H^tiimtes based on Finl Foiiow-up "urvey retpontcs. 

Notes: Sample size, classroom mining tubgn.jp: assignees = 1,058, control group ^ SI 3; OJT/JSA subgroup: 
assignees = 792. control group =: 368; other service* subgroup: assignees = 932, control group 3g5. Estimates 
arc regression- adjusted to control for differences in baseline characieri sties between the treatment group and 
control group; sec Appendix D. 

a. For columns I and 2 (*percenuge employed') this row shows the percentage of control gtxwp members who 
reported bdng employed at any time during the follow-up pehod and the cslimaied impact on this percentage 
for assignees, respectively. 

• Sutistically significant at the . 10 level. at the .05 level. at the .01 level (two-uilcd test). 



45S 
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Exhibit H,2I impacts on Total 18-Month Earnings: All Youth JTPA Assignees, 
by Gender and Ethnicity 



Impact, in S, adjiisied Jar 
sample distribution across: 





Sample 


Comrol 






Sites and service 


Subgroup 


size 


mean 


Impact, in $ 


Sites 


strategies 


0) 


(2) 


(3) 




(5) 


Female 


2,300 


S 6.202 


S-163 


S -236 


S -327 


Male 


1,748 


10.799 


-773 


-840** 


-939** 


F-tcsi, difference 












between genders 






n.s. 


n.s. 


n.s. 


Whites 


2.094 


S 9.628 


S -607* 


S-892** 


S -1.036** 


Nonwhites 


1.954 


6.776 


-238 


81 


47 


F-test, difference 












between genders 






n.s. 


n.s. 


m 



Source: £ai males based on First Follow-up Survey responses. 

Ndcs: Eslimaies are regrcssiorv-adjusiod lo control for differences in baseline charKaerislics brt v. ren the 
treatment group and control jroup; sec Appendix D. Control {roup means arc not rcgreafion-aJ uiicd. 
a This subgroup includes blacks. Hispanica. American Inlians. Alaskan Natives. A$i*ns, an! Pacific Islanders. 
* Siatislically significant at the .10 level. at the .05 level. at the ,01 level (F-test orlwo-uilod l-tcst); "n.s." 
means the F-test for the difference in in^acu between the subgroups in each let is not statistically significant . 



ExJubit H.22 Deviations of Impacts on the Total IS-hionth Earnings of Each Two-Year Youth 
Cohort from the Trend in Impacts on the IS-Month Earnings of Young Adults 
Ages 22 to 29: JTPA Assignees, by Gender and Service Strategy Subgroup 



Deviation of itnpact on 2-year youth cohort from 
Sajnple ^ trend in young adult impacts Qn $) ^ 



Service strategy- 
subgroup 


size" 
0) 


Age 16-17 

(2) 


Age 18-19 
(3) 


Age 20-21 
(4) 










Females 








Classroom training 


2 221 


S 


2.153 


S 


782 


S 


738 


OJT/JSA 


1.394 




-1.790 




-1.021 




118 


Other services 


1.152 




-643 




-441 




-2,884 


All subgroups 


4.767 




933 




211 




-169 










Males 








Classroom training 


1.050 


S 


-3.692 


S 


-3,995 


S 


-1 .075 


OJT/JSA 


1.608 




-h871 




-3.624 




-1.299 


Other services 


1.049 




194 




1.806 




677 


All subgroups 


3.707 




-1.912 




-2.188 




-702 



Source: Est i mates based on First FolIow>up Survey responses. 

Note: Esiimaics arc regression- adjusted to comrol for differences in baseline characteristics between the treaimeni 
grotjp and control group; see Appendix D. 

a Treatment and control groups combined, including all sample members who were ages 16 to 29 at random 
assignment. 

b NorK of the deviations in columns 2 through 4 was suiisiicaJly significant ai the . 10 level (two-iailcd le^i). 
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Exhibit H. 23 Impacts on th^ Number of Months qf Continuous Noneinployment between 
Random Assigntnent and the First Job after Random Assignment: Youth 
Treannent Group and Control Group » by Gender and Senice Strategy 
Subgroup 







Month of first job 


bnpact on 


nonemploytnent 




Sample 
size ^ 
(1) 


Treatment 


Control 


In 


As 7c 


Service strategy 
subgroup 


group 

aj 


group 


tnonths 
(4) 


of (3) 
(5) 








Female youths 






Classroom iraining 


1,W5 


5.5 


A.6 


0.9** 


I9.07t 


OJT/JSA 


545 


3.3 


4.3 


-1.1** 


-24.9 


Other services 


710 


4.5 


4.8 


.0.3 


-6.1 


All subgroups 


2.300 


4.6 


4.6 


0.0 


0.1 








Male youths 






Classroom training 


526 


4.6 


4.3 


0.3 


8.07c 


OJT/JSA 


615 


3.4 


3.1 


0.3 


9.8 


Other services 


607 


4.4 


3.7 


0.7 


18.6 


All subgroups 


1.748 


4.1 


3.7 


0.5* 


12.5 



Source: Estimates based on First Foliow'Up Surv^ responses. 
No(e: For the esiimation procedure see Appendix D. 
a. Treatment and control groups combined. 

♦ Sutisiically signiHcani aithc .10 level, ai the .05 level. at the .01 level (iwo-iailcd test). 
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